[
https://issues.apache.org/jira/browse/IMPALA-12434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell resolved IMPALA-12434.
------------------------------------
Fix Version/s: Impala 4.3.0
Resolution: Fixed
> Isolate shell/pkg_resources.py to keep it off of the PYTHONPATH
> ---------------------------------------------------------------
>
> Key: IMPALA-12434
> URL: https://issues.apache.org/jira/browse/IMPALA-12434
> Project: IMPALA
> Issue Type: Task
> Components: Infrastructure
> Affects Versions: Impala 4.3.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Major
> Fix For: Impala 4.3.0
>
>
> In some internal builds on Redhat 8, the pip install of impala-shell into the
> Python 3 virtualenv can fail with this error:
> {noformat}
> Exception:
> Traceback (most recent call last):
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/basecommand.py",
> line 215, in main
> status = self.run(options, args)
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/commands/install.py",
> line 357, in run
> wb.build(autobuilding=True)
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/wheel.py",
> line 753, in build
> self.requirement_set.prepare_files(self.finder)
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/req/req_set.py",
> line 381, in prepare_files
> ignore_dependencies=self.ignore_dependencies))
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/req/req_set.py",
> line 637, in _prepare_file
> abstract_dist.prep_for_dist()
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/req/req_set.py",
> line 130, in prep_for_dist
> self.req_to_install.run_egg_info()
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/req/req_install.py",
> line 417, in run_egg_info
> self.setup_py, self.link,
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/req/req_install.py",
> line 387, in setup_py
> import setuptools # noqa
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/setuptools/__init__.py",
> line 12, in <module>
> import setuptools.version
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/setuptools/version.py",
> line 1, in <module>
> import pkg_resources
> File "impala/shell/pkg_resources.py", line 50
> def _bypass_ensure_directory(name, mode=0777):
> ^
> SyntaxError: invalid token
> make[3]: *** [shell/CMakeFiles/shell_python3_install.dir/build.make:70:
> shell/CMakeFiles/shell_python3_install] Error 2
> {noformat}
> What seems to happen is shell/pkg_resources.py is on the PYTHONPATH and our
> pkg_resources.py doesn't work with Python 3. If I move pkg_resources.py to a
> different directory (like a legacy subdirectory) or if I add a
> WORKING_DIRECTORY for the CMake command, it works fine. In general, keeping
> pkg_resources.py away from our PYTHONPATH (which includes $IMPALA_HOME/shell)
> seems like a good idea.
> This problem doesn't reproduce on the regular upstream Impala build or on my
> local Ubuntu 20 workstation build, but it reliably reproduces on an internal
> build.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
