[ https://issues.apache.org/jira/browse/IMPALA-12434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joe McDonnell resolved IMPALA-12434. ------------------------------------ Fix Version/s: Impala 4.3.0 Resolution: Fixed > Isolate shell/pkg_resources.py to keep it off of the PYTHONPATH > --------------------------------------------------------------- > > Key: IMPALA-12434 > URL: https://issues.apache.org/jira/browse/IMPALA-12434 > Project: IMPALA > Issue Type: Task > Components: Infrastructure > Affects Versions: Impala 4.3.0 > Reporter: Joe McDonnell > Assignee: Joe McDonnell > Priority: Major > Fix For: Impala 4.3.0 > > > In some internal builds on Redhat 8, the pip install of impala-shell into the > Python 3 virtualenv can fail with this error: > {noformat} > Exception: > Traceback (most recent call last): > File > "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/basecommand.py", > line 215, in main > status = self.run(options, args) > File > "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/commands/install.py", > line 357, in run > wb.build(autobuilding=True) > File > "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/wheel.py", > line 753, in build > self.requirement_set.prepare_files(self.finder) > File > "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/req/req_set.py", > line 381, in prepare_files > ignore_dependencies=self.ignore_dependencies)) > File > "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/req/req_set.py", > line 637, in _prepare_file > abstract_dist.prep_for_dist() > File > "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/req/req_set.py", > line 130, in prep_for_dist > self.req_to_install.run_egg_info() > File > "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/req/req_install.py", > line 417, in run_egg_info > self.setup_py, self.link, > File > "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/req/req_install.py", > line 387, in setup_py > import setuptools # noqa > File > "impala/shell/build/py3_venv/lib64/python3.6/site-packages/setuptools/__init__.py", > line 12, in <module> > import setuptools.version > File > "impala/shell/build/py3_venv/lib64/python3.6/site-packages/setuptools/version.py", > line 1, in <module> > import pkg_resources > File "impala/shell/pkg_resources.py", line 50 > def _bypass_ensure_directory(name, mode=0777): > ^ > SyntaxError: invalid token > make[3]: *** [shell/CMakeFiles/shell_python3_install.dir/build.make:70: > shell/CMakeFiles/shell_python3_install] Error 2 > {noformat} > What seems to happen is shell/pkg_resources.py is on the PYTHONPATH and our > pkg_resources.py doesn't work with Python 3. If I move pkg_resources.py to a > different directory (like a legacy subdirectory) or if I add a > WORKING_DIRECTORY for the CMake command, it works fine. In general, keeping > pkg_resources.py away from our PYTHONPATH (which includes $IMPALA_HOME/shell) > seems like a good idea. > This problem doesn't reproduce on the regular upstream Impala build or on my > local Ubuntu 20 workstation build, but it reliably reproduces on an internal > build. -- This message was sent by Atlassian Jira (v8.20.10#820010)