Alibm73 opened a new issue #12547: URL: https://github.com/apache/airflow/issues/12547
**Apache Airflow version**: 1.10.12 **Environment**: VMware VM **Cloud provider or hardware configuration**: 8 G RAM, 50G HDD, 4Cores **OS**: NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/"; BUG_REPORT_URL="https://bugs.centos.org/"; CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7" **Kernel**: Linux mycentosvm 3.10.0-1160.6.1.el7.x86_64 #1 SMP Tue Nov 17 13:59:11 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux **Install tools**: yum groupinstall "Development tools" yum install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel python3-devel wget gcc gcc-c++ libffi-devel cyrus-sasl-devel yum install mysql-server [version8] [set global_explicit, create airflow database and user then give permissions] alternative python 3.7 [pip 20.2.4 setuptools 47.1.0 ] **What happened**: All kinds of errors: I copied ```https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.7.txt``` to a file called ```req.txt``` to be able to change the dependencies 1. Error1: ``` ERROR: Could not satisfy constraints for 'pyhive': installation from path or url cannot be constrained to a version ``` My solution1: Install specified pyhive manually ``` pip install PyHive==0.6.3 ``` 2. Error2: ``` ERROR: Cannot install apache-airflow, apache-airflow and apache-airflow because these package versions have conflicting dependencies. The conflict is caused by: email-validator 1.1.1 depends on idna>=2.0.0 requests 2.24.0 depends on idna<3 and >=2.5 moto 1.3.14 depends on idna<2.9 and >=2.5 To fix this you could try to: 1. loosen the range of package versions you've specified 2. remove package versions to allow pip attempt to solve the dependency conflict ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies ``` My solution2: ``` pip install idna==2.9 ``` 3. Error3: ``` ERROR: Cannot install apache-airflow[async,celery,devel,devel-hadoop,druid,hdfs,hive,jdbc,mysql,rabbitmq,ssh] 1.10.12, apache-airflow and apache-airflow because these package versions have conflicting dependencies. The conflict is caused by: apache-airflow[async,celery,devel,devel-hadoop,druid,hdfs,hive,jdbc,mysql,rabbitmq,ssh] 1.10.12 depends on pytest<6.0.0; extra == "devel_hadoop" pytest-rerunfailures 9.0 depends on pytest>=5.0 pytest-xdist 2.0.0 depends on pytest>=6.0.0 To fix this you could try to: 1. loosen the range of package versions you've specified 2. remove package versions to allow pip attempt to solve the dependency conflict ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies ``` My solution3: ``` pip install pytest==6.0.0 ``` 4. Error4: ``` Downloading eventlet-0.14.0.tar.gz (279 kB) |████████████████████████████████| 279 kB 404 kB/s ERROR: Command errored out with exit status 1: command: /usr/local/bin/python3.7 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ozxuwjov/eventlet_711991891ddc448ba8dd97630910301c/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ozxuwjov/eventlet_711991891ddc448ba8dd97630910301c/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-aw4qpa8h cwd: /tmp/pip-install-ozxuwjov/eventlet_711991891ddc448ba8dd97630910301c/ Complete output (8 lines): Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/pip-install-ozxuwjov/eventlet_711991891ddc448ba8dd97630910301c/setup.py", line 3, in <module> from eventlet import __version__ File "/tmp/pip-install-ozxuwjov/eventlet_711991891ddc448ba8dd97630910301c/eventlet/__init__.py", line 42 except ImportError, e: ^ SyntaxError: invalid syntax ---------------------------------------- ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output. ``` My solution4: ``` pip install eventlet==0.29.1 ``` 5. Error5: ``` bitmq,ssh]==1.10.12) (0.4.17) ERROR: Could not find a version that satisfies the requirement pytest<6.0.0; extra == "devel_hadoop" (from apache-airflow[async,celery,devel,devel-hadoop,druid,hdfs,hive,jdbc,mysql,rabbitmq,ssh]) ERROR: No matching distribution found for pytest<6.0.0; extra == "devel_hadoop" ``` then install older version and i get this error: ``` Downloading paramiko-2.7.1-py2.py3-none-any.whl (206 kB) |████████████████████████████████| 206 kB 377 kB/s Collecting pytest-xdist Using cached pytest_xdist-2.0.0-py2.py3-none-any.whl (36 kB) ERROR: Cannot install apache-airflow[async,celery,devel,devel-hadoop,druid,hdfs,hive,jdbc,mysql,rabbitmq,ssh] 1.10.12 and apache-airflow because these package versions have conflicting dependencies. The conflict is caused by: apache-airflow[async,celery,devel,devel-hadoop,druid,hdfs,hive,jdbc,mysql,rabbitmq,ssh] 1.10.12 depends on pytest<6.0.0; extra == "devel_hadoop" pytest-xdist 2.0.0 depends on pytest>=6.0.0 To fix this you could try to: 1. loosen the range of package versions you've specified 2. remove package versions to allow pip attempt to solve the dependency conflict ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies ``` I even don't know any of my solutions even work! **How to reproduce it**: Run this command ```sh pip install \ apache-airflow[async,celery,devel,devel_hadoop,druid,hdfs,hive,jdbc,mysql,rabbitmq,ssh]==1.10.12 \ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.7.txt"; \ --use-feature=2020-resolver ``` **Anything else we need to know**: According to the announcement linked below, pip will introduce a new dependency resolver in October 2020, which will be more robust but might break some existing setups. Therefore they are suggesting users to try running their pip install scripts at least once (in dev mode) with this option: --use-feature=2020-resolver to anticipate any potential issue before the new resolver becomes the default in October 2020 with pip version 20.3. [](https://discuss.python.org/t/announcement-pip-20-2-release/4863) How often does this problem occur? Once? Every time etc? Everytime, the dependency problem is a nightmare, especially with adding hadoop extras ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
