[Impala-ASF-CR] IMPALA-11399: Download shell dependencies from PyPI

2022-08-20 Thread Michael Smith (Code Review)
Michael Smith has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18668 )

Change subject: IMPALA-11399: Download shell dependencies from PyPI
..


Patch Set 10:

For the moment, this would need a patch applied to thrift_sasl's setup.py after 
download to make it work on older CentOS 7 systems, or use toolchain python to 
create the Python 2 virtualenv.


--
To view, visit http://gerrit.cloudera.org:8080/18668
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b76e112ba9d0db19fae3e8eb15fd54a721f80fd
Gerrit-Change-Number: 18668
Gerrit-PatchSet: 10
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Mon, 15 Aug 2022 20:25:29 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11399: Download shell dependencies from PyPI

2022-08-11 Thread Michael Smith (Code Review)
Michael Smith has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18668 )

Change subject: IMPALA-11399: Download shell dependencies from PyPI
..


Patch Set 10:

I ran tests on CentOS 7 to check for any issues with the setup for thrift_sasl 
(which was previously patched). It built successfully.


--
To view, visit http://gerrit.cloudera.org:8080/18668
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b76e112ba9d0db19fae3e8eb15fd54a721f80fd
Gerrit-Change-Number: 18668
Gerrit-PatchSet: 10
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Thu, 11 Aug 2022 16:16:09 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11399: Download shell dependencies from PyPI

2022-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18668 )

Change subject: IMPALA-11399: Download shell dependencies from PyPI
..


Patch Set 10: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/18668
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b76e112ba9d0db19fae3e8eb15fd54a721f80fd
Gerrit-Change-Number: 18668
Gerrit-PatchSet: 10
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Wed, 10 Aug 2022 22:57:13 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11399: Download shell dependencies from PyPI

2022-08-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18668 )

Change subject: IMPALA-11399: Download shell dependencies from PyPI
..


Patch Set 10:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8430/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/18668
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b76e112ba9d0db19fae3e8eb15fd54a721f80fd
Gerrit-Change-Number: 18668
Gerrit-PatchSet: 10
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Wed, 10 Aug 2022 17:57:54 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11399: Download shell dependencies from PyPI

2022-07-14 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18668 )

Change subject: IMPALA-11399: Download shell dependencies from PyPI
..


Patch Set 8:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/10971/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/18668
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b76e112ba9d0db19fae3e8eb15fd54a721f80fd
Gerrit-Change-Number: 18668
Gerrit-PatchSet: 8
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Thu, 14 Jul 2022 17:04:44 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11399: Download shell dependencies from PyPI

2022-07-14 Thread Michael Smith (Code Review)
Hello Joe McDonnell, Wenzhe Zhou, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/18668

to look at the new patch set (#8).

Change subject: IMPALA-11399: Download shell dependencies from PyPI
..

IMPALA-11399: Download shell dependencies from PyPI

Downloads shell dependencies from PyPI rather than including them in the
repository. Simplifies dependency updates so we only need to update
requirements.txt.

Removes old dependencies from requirements.txt. configparser is included
in all versions of Python we support, and setuptools is generally
installed as a system package. PyPI configparser does include Python 3
extensions over Python 2's ConfigParser, but since we didn't include
configparser in the tarball I conclude we don't use those extensions.

Change-Id: I3b76e112ba9d0db19fae3e8eb15fd54a721f80fd
---
M bin/impala-config.sh
M infra/python/deps/download_requirements
M infra/python/deps/find_py26.py
M infra/python/deps/pip_download.py
M shell/.gitignore
D shell/ext-py/bitarray-2.3.0/CHANGE_LOG
D shell/ext-py/bitarray-2.3.0/LICENSE
D shell/ext-py/bitarray-2.3.0/README.rst
D shell/ext-py/bitarray-2.3.0/bitarray/__init__.py
D shell/ext-py/bitarray-2.3.0/bitarray/__init__.pyi
D shell/ext-py/bitarray-2.3.0/bitarray/_bitarray.c
D shell/ext-py/bitarray-2.3.0/bitarray/_util.c
D shell/ext-py/bitarray-2.3.0/bitarray/architecture.txt
D shell/ext-py/bitarray-2.3.0/bitarray/bitarray.h
D shell/ext-py/bitarray-2.3.0/bitarray/copy_n.txt
D shell/ext-py/bitarray-2.3.0/bitarray/py.typed
D shell/ext-py/bitarray-2.3.0/bitarray/pythoncapi_compat.h
D shell/ext-py/bitarray-2.3.0/bitarray/test_bitarray.py
D shell/ext-py/bitarray-2.3.0/bitarray/test_data.pickle
D shell/ext-py/bitarray-2.3.0/bitarray/test_util.py
D shell/ext-py/bitarray-2.3.0/bitarray/util.py
D shell/ext-py/bitarray-2.3.0/bitarray/util.pyi
D shell/ext-py/bitarray-2.3.0/contributing.md
D shell/ext-py/bitarray-2.3.0/setup.py
D shell/ext-py/bitarray-2.3.0/update_doc.py
D shell/ext-py/kerberos-1.3.1/MANIFEST.in
D shell/ext-py/kerberos-1.3.1/PKG-INFO
D shell/ext-py/kerberos-1.3.1/README.md
D shell/ext-py/kerberos-1.3.1/pysrc/kerberos.py
D shell/ext-py/kerberos-1.3.1/setup.cfg
D shell/ext-py/kerberos-1.3.1/setup.py
D shell/ext-py/kerberos-1.3.1/src/base64.c
D shell/ext-py/kerberos-1.3.1/src/base64.h
D shell/ext-py/kerberos-1.3.1/src/kerberos.c
D shell/ext-py/kerberos-1.3.1/src/kerberosbasic.c
D shell/ext-py/kerberos-1.3.1/src/kerberosbasic.h
D shell/ext-py/kerberos-1.3.1/src/kerberosgss.c
D shell/ext-py/kerberos-1.3.1/src/kerberosgss.h
D shell/ext-py/kerberos-1.3.1/src/kerberospw.c
D shell/ext-py/kerberos-1.3.1/src/kerberospw.h
D shell/ext-py/prettytable-0.7.2/CHANGELOG
D shell/ext-py/prettytable-0.7.2/COPYING
D shell/ext-py/prettytable-0.7.2/MANIFEST.in
D shell/ext-py/prettytable-0.7.2/PKG-INFO
D shell/ext-py/prettytable-0.7.2/README
D shell/ext-py/prettytable-0.7.2/prettytable.py
D shell/ext-py/prettytable-0.7.2/setup.cfg
D shell/ext-py/prettytable-0.7.2/setup.py
D shell/ext-py/sasl-0.2.1/LICENSE.txt
D shell/ext-py/sasl-0.2.1/MANIFEST.in
D shell/ext-py/sasl-0.2.1/recython.sh
D shell/ext-py/sasl-0.2.1/sasl/__init__.py
D shell/ext-py/sasl-0.2.1/sasl/saslwrapper.cpp
D shell/ext-py/sasl-0.2.1/sasl/saslwrapper.h
D shell/ext-py/sasl-0.2.1/sasl/saslwrapper.pyx
D shell/ext-py/sasl-0.2.1/setup.py
D shell/ext-py/six-1.14.0/CHANGES
D shell/ext-py/six-1.14.0/CONTRIBUTORS
D shell/ext-py/six-1.14.0/LICENSE
D shell/ext-py/six-1.14.0/MANIFEST.in
D shell/ext-py/six-1.14.0/README.rst
D shell/ext-py/six-1.14.0/setup.cfg
D shell/ext-py/six-1.14.0/setup.py
D shell/ext-py/six-1.14.0/six.py
D shell/ext-py/six-1.14.0/test_six.py
D shell/ext-py/six-1.14.0/tox.ini
D shell/ext-py/sqlparse-0.3.1/AUTHORS
D shell/ext-py/sqlparse-0.3.1/CHANGELOG
D shell/ext-py/sqlparse-0.3.1/LICENSE
D shell/ext-py/sqlparse-0.3.1/MANIFEST.in
D shell/ext-py/sqlparse-0.3.1/README.rst
D shell/ext-py/sqlparse-0.3.1/TODO
D shell/ext-py/sqlparse-0.3.1/setup.cfg
D shell/ext-py/sqlparse-0.3.1/setup.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/__init__.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/__main__.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/cli.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/compat.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/engine/__init__.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/engine/filter_stack.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/engine/grouping.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/engine/statement_splitter.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/exceptions.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/__init__.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/aligned_indent.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/others.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/output.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/reindent.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/right_margin.py
D 

[Impala-ASF-CR] IMPALA-11399: Download shell dependencies from PyPI

2022-07-14 Thread Michael Smith (Code Review)
Michael Smith has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18668 )

Change subject: IMPALA-11399: Download shell dependencies from PyPI
..


Patch Set 7:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/18668/7/infra/python/deps/download_requirements
File infra/python/deps/download_requirements:

http://gerrit.cloudera.org:8080/#/c/18668/7/infra/python/deps/download_requirements@27
PS7, Line 27: *requirements.txt
> Nit: I'm conflicted about having a wildcard vs just listing the specific re
Ok, that's easy enough to add back.


http://gerrit.cloudera.org:8080/#/c/18668/7/shell/make_shell_tarball.sh
File shell/make_shell_tarball.sh:

http://gerrit.cloudera.org:8080/#/c/18668/7/shell/make_shell_tarball.sh@107
PS7, Line 107: rm -rf ${SHELL_HOME}/ext-py && mkdir -p ${SHELL_HOME}/ext-py
 : for line in $(cat ${SHELL_HOME}/packaging/requirements.txt); do
 :   package=${line%==*}
 :   version=${line#*==}
 :   module=${SHELL_HOME}/packaging/${package}-${version}
 :   echo "Extracting ${module}"
 :   if [ -f ${module}.tar.gz ]; then
 : tar xzf ${module}.tar.gz -C ${SHELL_HOME}/ext-py
 :   else
 : tar xjf ${module}.tar.bz2 -C ${SHELL_HOME}/ext-py
> I'm concerned about putting the the unpacked tarball files in ${IMPALA_HOME
That makes some sense, although it shouldn't really happen because specific 
versions should be fixed artifacts. I can unpack them to build/ instead.


http://gerrit.cloudera.org:8080/#/c/18668/7/shell/packaging/requirements.txt
File shell/packaging/requirements.txt:

http://gerrit.cloudera.org:8080/#/c/18668/7/shell/packaging/requirements.txt@8
PS7, Line 8: thrift_sasl==0.4.3
> One complication is that we patched thrift_sasl to allow things to build on
We've updated CentOS environments recently. This is primarily a build-time 
issue, so if it passes the private parameterized build it might be fine.



--
To view, visit http://gerrit.cloudera.org:8080/18668
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b76e112ba9d0db19fae3e8eb15fd54a721f80fd
Gerrit-Change-Number: 18668
Gerrit-PatchSet: 7
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Thu, 14 Jul 2022 16:15:29 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11399: Download shell dependencies from PyPI

2022-07-12 Thread Joe McDonnell (Code Review)
Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18668 )

Change subject: IMPALA-11399: Download shell dependencies from PyPI
..


Patch Set 7:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/18668/7/infra/python/deps/download_requirements
File infra/python/deps/download_requirements:

http://gerrit.cloudera.org:8080/#/c/18668/7/infra/python/deps/download_requirements@27
PS7, Line 27: *requirements.txt
Nit: I'm conflicted about having a wildcard vs just listing the specific 
requirements.txt files.


http://gerrit.cloudera.org:8080/#/c/18668/7/shell/make_shell_tarball.sh
File shell/make_shell_tarball.sh:

http://gerrit.cloudera.org:8080/#/c/18668/7/shell/make_shell_tarball.sh@107
PS7, Line 107: rm -rf ${SHELL_HOME}/ext-py && mkdir -p ${SHELL_HOME}/ext-py
 : for line in $(cat ${SHELL_HOME}/packaging/requirements.txt); do
 :   package=${line%==*}
 :   version=${line#*==}
 :   module=${SHELL_HOME}/packaging/${package}-${version}
 :   echo "Extracting ${module}"
 :   if [ -f ${module}.tar.gz ]; then
 : tar xzf ${module}.tar.gz -C ${SHELL_HOME}/ext-py
 :   else
 : tar xjf ${module}.tar.bz2 -C ${SHELL_HOME}/ext-py
I'm concerned about putting the the unpacked tarball files in 
${IMPALA_HOME}/shell/ext-py, because when switching branches the leftover files 
could conflict with our older branches that have git-tracked files there.

Is that an actual problem or does it work?

I like the idea of an out-of-tree build where anything not part of the source 
is located in a separate directory (like our build/ directory).


http://gerrit.cloudera.org:8080/#/c/18668/7/shell/packaging/requirements.txt
File shell/packaging/requirements.txt:

http://gerrit.cloudera.org:8080/#/c/18668/7/shell/packaging/requirements.txt@8
PS7, Line 8: thrift_sasl==0.4.3
One complication is that we patched thrift_sasl to allow things to build on old 
Centos 7 environments:
https://github.com/apache/impala/commit/1a61a8025c87c37921a1bba4c49f754d8bd10bcc

Switching to downloading from pypi might reintroduce that issue. We've been 
meaning to fix it so that we can remove the patch. That is tracked in 
IMPALA-10941. I think it is a doable thing, but it has been low priority.



--
To view, visit http://gerrit.cloudera.org:8080/18668
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b76e112ba9d0db19fae3e8eb15fd54a721f80fd
Gerrit-Change-Number: 18668
Gerrit-PatchSet: 7
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Wed, 13 Jul 2022 01:23:51 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-11399: Download shell dependencies from PyPI

2022-06-30 Thread Wenzhe Zhou (Code Review)
Wenzhe Zhou has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18668 )

Change subject: IMPALA-11399: Download shell dependencies from PyPI
..


Patch Set 7: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/18668
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b76e112ba9d0db19fae3e8eb15fd54a721f80fd
Gerrit-Change-Number: 18668
Gerrit-PatchSet: 7
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Fri, 01 Jul 2022 00:53:19 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11399: Download shell dependencies from PyPI

2022-06-28 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18668 )

Change subject: IMPALA-11399: Download shell dependencies from PyPI
..


Patch Set 7: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/18668
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b76e112ba9d0db19fae3e8eb15fd54a721f80fd
Gerrit-Change-Number: 18668
Gerrit-PatchSet: 7
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Smith 
Gerrit-Comment-Date: Tue, 28 Jun 2022 22:27:55 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11399: Download shell dependencies from PyPI

2022-06-28 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18668 )

Change subject: IMPALA-11399: Download shell dependencies from PyPI
..


Patch Set 7:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8276/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/18668
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b76e112ba9d0db19fae3e8eb15fd54a721f80fd
Gerrit-Change-Number: 18668
Gerrit-PatchSet: 7
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Smith 
Gerrit-Comment-Date: Tue, 28 Jun 2022 17:35:22 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11399: Download shell dependencies from PyPI

2022-06-28 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18668 )

Change subject: IMPALA-11399: Download shell dependencies from PyPI
..


Patch Set 7:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/10892/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/18668
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b76e112ba9d0db19fae3e8eb15fd54a721f80fd
Gerrit-Change-Number: 18668
Gerrit-PatchSet: 7
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Smith 
Gerrit-Comment-Date: Tue, 28 Jun 2022 17:31:49 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11399: Download shell dependencies from PyPI

2022-06-28 Thread Michael Smith (Code Review)
Hello Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/18668

to look at the new patch set (#7).

Change subject: IMPALA-11399: Download shell dependencies from PyPI
..

IMPALA-11399: Download shell dependencies from PyPI

Downloads shell dependencies from PyPI rather than including them in the
repository. Simplifies dependency updates so we only need to update
requirements.txt.

Removes old dependencies from requirements.txt. configparser is included
in all versions of Python we support, and setuptools is generally
installed as a system package. PyPI configparser does include Python 3
extensions over Python 2's ConfigParser, but since we didn't include
configparser in the tarball I conclude we don't use those extensions.

Change-Id: I3b76e112ba9d0db19fae3e8eb15fd54a721f80fd
---
M bin/impala-config.sh
M infra/python/deps/download_requirements
M infra/python/deps/find_py26.py
M infra/python/deps/pip_download.py
M shell/.gitignore
D shell/ext-py/bitarray-2.3.0/CHANGE_LOG
D shell/ext-py/bitarray-2.3.0/LICENSE
D shell/ext-py/bitarray-2.3.0/README.rst
D shell/ext-py/bitarray-2.3.0/bitarray/__init__.py
D shell/ext-py/bitarray-2.3.0/bitarray/__init__.pyi
D shell/ext-py/bitarray-2.3.0/bitarray/_bitarray.c
D shell/ext-py/bitarray-2.3.0/bitarray/_util.c
D shell/ext-py/bitarray-2.3.0/bitarray/architecture.txt
D shell/ext-py/bitarray-2.3.0/bitarray/bitarray.h
D shell/ext-py/bitarray-2.3.0/bitarray/copy_n.txt
D shell/ext-py/bitarray-2.3.0/bitarray/py.typed
D shell/ext-py/bitarray-2.3.0/bitarray/pythoncapi_compat.h
D shell/ext-py/bitarray-2.3.0/bitarray/test_bitarray.py
D shell/ext-py/bitarray-2.3.0/bitarray/test_data.pickle
D shell/ext-py/bitarray-2.3.0/bitarray/test_util.py
D shell/ext-py/bitarray-2.3.0/bitarray/util.py
D shell/ext-py/bitarray-2.3.0/bitarray/util.pyi
D shell/ext-py/bitarray-2.3.0/contributing.md
D shell/ext-py/bitarray-2.3.0/setup.py
D shell/ext-py/bitarray-2.3.0/update_doc.py
D shell/ext-py/kerberos-1.3.1/MANIFEST.in
D shell/ext-py/kerberos-1.3.1/PKG-INFO
D shell/ext-py/kerberos-1.3.1/README.md
D shell/ext-py/kerberos-1.3.1/pysrc/kerberos.py
D shell/ext-py/kerberos-1.3.1/setup.cfg
D shell/ext-py/kerberos-1.3.1/setup.py
D shell/ext-py/kerberos-1.3.1/src/base64.c
D shell/ext-py/kerberos-1.3.1/src/base64.h
D shell/ext-py/kerberos-1.3.1/src/kerberos.c
D shell/ext-py/kerberos-1.3.1/src/kerberosbasic.c
D shell/ext-py/kerberos-1.3.1/src/kerberosbasic.h
D shell/ext-py/kerberos-1.3.1/src/kerberosgss.c
D shell/ext-py/kerberos-1.3.1/src/kerberosgss.h
D shell/ext-py/kerberos-1.3.1/src/kerberospw.c
D shell/ext-py/kerberos-1.3.1/src/kerberospw.h
D shell/ext-py/prettytable-0.7.2/CHANGELOG
D shell/ext-py/prettytable-0.7.2/COPYING
D shell/ext-py/prettytable-0.7.2/MANIFEST.in
D shell/ext-py/prettytable-0.7.2/PKG-INFO
D shell/ext-py/prettytable-0.7.2/README
D shell/ext-py/prettytable-0.7.2/prettytable.py
D shell/ext-py/prettytable-0.7.2/setup.cfg
D shell/ext-py/prettytable-0.7.2/setup.py
D shell/ext-py/sasl-0.2.1/LICENSE.txt
D shell/ext-py/sasl-0.2.1/MANIFEST.in
D shell/ext-py/sasl-0.2.1/recython.sh
D shell/ext-py/sasl-0.2.1/sasl/__init__.py
D shell/ext-py/sasl-0.2.1/sasl/saslwrapper.cpp
D shell/ext-py/sasl-0.2.1/sasl/saslwrapper.h
D shell/ext-py/sasl-0.2.1/sasl/saslwrapper.pyx
D shell/ext-py/sasl-0.2.1/setup.py
D shell/ext-py/six-1.14.0/CHANGES
D shell/ext-py/six-1.14.0/CONTRIBUTORS
D shell/ext-py/six-1.14.0/LICENSE
D shell/ext-py/six-1.14.0/MANIFEST.in
D shell/ext-py/six-1.14.0/README.rst
D shell/ext-py/six-1.14.0/setup.cfg
D shell/ext-py/six-1.14.0/setup.py
D shell/ext-py/six-1.14.0/six.py
D shell/ext-py/six-1.14.0/test_six.py
D shell/ext-py/six-1.14.0/tox.ini
D shell/ext-py/sqlparse-0.3.1/AUTHORS
D shell/ext-py/sqlparse-0.3.1/CHANGELOG
D shell/ext-py/sqlparse-0.3.1/LICENSE
D shell/ext-py/sqlparse-0.3.1/MANIFEST.in
D shell/ext-py/sqlparse-0.3.1/README.rst
D shell/ext-py/sqlparse-0.3.1/TODO
D shell/ext-py/sqlparse-0.3.1/setup.cfg
D shell/ext-py/sqlparse-0.3.1/setup.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/__init__.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/__main__.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/cli.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/compat.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/engine/__init__.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/engine/filter_stack.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/engine/grouping.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/engine/statement_splitter.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/exceptions.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/__init__.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/aligned_indent.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/others.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/output.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/reindent.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/right_margin.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/tokens.py
D 

[Impala-ASF-CR] IMPALA-11399: Download shell dependencies from PyPI

2022-06-27 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18668 )

Change subject: IMPALA-11399: Download shell dependencies from PyPI
..


Patch Set 6: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8274/


--
To view, visit http://gerrit.cloudera.org:8080/18668
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b76e112ba9d0db19fae3e8eb15fd54a721f80fd
Gerrit-Change-Number: 18668
Gerrit-PatchSet: 6
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Smith 
Gerrit-Comment-Date: Mon, 27 Jun 2022 21:30:12 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11399: Download shell dependencies from PyPI

2022-06-27 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18668 )

Change subject: IMPALA-11399: Download shell dependencies from PyPI
..


Patch Set 6:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/10886/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/18668
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b76e112ba9d0db19fae3e8eb15fd54a721f80fd
Gerrit-Change-Number: 18668
Gerrit-PatchSet: 6
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Smith 
Gerrit-Comment-Date: Mon, 27 Jun 2022 16:28:04 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11399: Download shell dependencies from PyPI

2022-06-27 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18668 )

Change subject: IMPALA-11399: Download shell dependencies from PyPI
..


Patch Set 6:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8274/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/18668
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b76e112ba9d0db19fae3e8eb15fd54a721f80fd
Gerrit-Change-Number: 18668
Gerrit-PatchSet: 6
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Smith 
Gerrit-Comment-Date: Mon, 27 Jun 2022 16:10:01 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-11399: Download shell dependencies from PyPI

2022-06-27 Thread Michael Smith (Code Review)
Hello Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/18668

to look at the new patch set (#6).

Change subject: IMPALA-11399: Download shell dependencies from PyPI
..

IMPALA-11399: Download shell dependencies from PyPI

Downloads shell dependencies from PyPI rather than including them in the
repository. Simplifies dependency updates so we only need to update
requirements.txt.

Removes old dependencies from requirements.txt. configparser is included
in all versions of Python we support, and setuptools is generally
installed as a system package. PyPI configparser does include Python 3
extensions over Python 2's ConfigParser, but since we didn't include
configparser in the tarball I conclude we don't use those extensions.

Change-Id: I3b76e112ba9d0db19fae3e8eb15fd54a721f80fd
---
M bin/impala-config.sh
M infra/python/deps/download_requirements
M infra/python/deps/find_py26.py
M infra/python/deps/pip_download.py
M shell/.gitignore
D shell/ext-py/bitarray-2.3.0/CHANGE_LOG
D shell/ext-py/bitarray-2.3.0/LICENSE
D shell/ext-py/bitarray-2.3.0/README.rst
D shell/ext-py/bitarray-2.3.0/bitarray/__init__.py
D shell/ext-py/bitarray-2.3.0/bitarray/__init__.pyi
D shell/ext-py/bitarray-2.3.0/bitarray/_bitarray.c
D shell/ext-py/bitarray-2.3.0/bitarray/_util.c
D shell/ext-py/bitarray-2.3.0/bitarray/architecture.txt
D shell/ext-py/bitarray-2.3.0/bitarray/bitarray.h
D shell/ext-py/bitarray-2.3.0/bitarray/copy_n.txt
D shell/ext-py/bitarray-2.3.0/bitarray/py.typed
D shell/ext-py/bitarray-2.3.0/bitarray/pythoncapi_compat.h
D shell/ext-py/bitarray-2.3.0/bitarray/test_bitarray.py
D shell/ext-py/bitarray-2.3.0/bitarray/test_data.pickle
D shell/ext-py/bitarray-2.3.0/bitarray/test_util.py
D shell/ext-py/bitarray-2.3.0/bitarray/util.py
D shell/ext-py/bitarray-2.3.0/bitarray/util.pyi
D shell/ext-py/bitarray-2.3.0/contributing.md
D shell/ext-py/bitarray-2.3.0/setup.py
D shell/ext-py/bitarray-2.3.0/update_doc.py
D shell/ext-py/kerberos-1.3.1/MANIFEST.in
D shell/ext-py/kerberos-1.3.1/PKG-INFO
D shell/ext-py/kerberos-1.3.1/README.md
D shell/ext-py/kerberos-1.3.1/pysrc/kerberos.py
D shell/ext-py/kerberos-1.3.1/setup.cfg
D shell/ext-py/kerberos-1.3.1/setup.py
D shell/ext-py/kerberos-1.3.1/src/base64.c
D shell/ext-py/kerberos-1.3.1/src/base64.h
D shell/ext-py/kerberos-1.3.1/src/kerberos.c
D shell/ext-py/kerberos-1.3.1/src/kerberosbasic.c
D shell/ext-py/kerberos-1.3.1/src/kerberosbasic.h
D shell/ext-py/kerberos-1.3.1/src/kerberosgss.c
D shell/ext-py/kerberos-1.3.1/src/kerberosgss.h
D shell/ext-py/kerberos-1.3.1/src/kerberospw.c
D shell/ext-py/kerberos-1.3.1/src/kerberospw.h
D shell/ext-py/prettytable-0.7.2/CHANGELOG
D shell/ext-py/prettytable-0.7.2/COPYING
D shell/ext-py/prettytable-0.7.2/MANIFEST.in
D shell/ext-py/prettytable-0.7.2/PKG-INFO
D shell/ext-py/prettytable-0.7.2/README
D shell/ext-py/prettytable-0.7.2/prettytable.py
D shell/ext-py/prettytable-0.7.2/setup.cfg
D shell/ext-py/prettytable-0.7.2/setup.py
D shell/ext-py/sasl-0.2.1/LICENSE.txt
D shell/ext-py/sasl-0.2.1/MANIFEST.in
D shell/ext-py/sasl-0.2.1/recython.sh
D shell/ext-py/sasl-0.2.1/sasl/__init__.py
D shell/ext-py/sasl-0.2.1/sasl/saslwrapper.cpp
D shell/ext-py/sasl-0.2.1/sasl/saslwrapper.h
D shell/ext-py/sasl-0.2.1/sasl/saslwrapper.pyx
D shell/ext-py/sasl-0.2.1/setup.py
D shell/ext-py/six-1.14.0/CHANGES
D shell/ext-py/six-1.14.0/CONTRIBUTORS
D shell/ext-py/six-1.14.0/LICENSE
D shell/ext-py/six-1.14.0/MANIFEST.in
D shell/ext-py/six-1.14.0/README.rst
D shell/ext-py/six-1.14.0/setup.cfg
D shell/ext-py/six-1.14.0/setup.py
D shell/ext-py/six-1.14.0/six.py
D shell/ext-py/six-1.14.0/test_six.py
D shell/ext-py/six-1.14.0/tox.ini
D shell/ext-py/sqlparse-0.3.1/AUTHORS
D shell/ext-py/sqlparse-0.3.1/CHANGELOG
D shell/ext-py/sqlparse-0.3.1/LICENSE
D shell/ext-py/sqlparse-0.3.1/MANIFEST.in
D shell/ext-py/sqlparse-0.3.1/README.rst
D shell/ext-py/sqlparse-0.3.1/TODO
D shell/ext-py/sqlparse-0.3.1/setup.cfg
D shell/ext-py/sqlparse-0.3.1/setup.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/__init__.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/__main__.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/cli.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/compat.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/engine/__init__.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/engine/filter_stack.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/engine/grouping.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/engine/statement_splitter.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/exceptions.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/__init__.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/aligned_indent.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/others.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/output.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/reindent.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/right_margin.py
D shell/ext-py/sqlparse-0.3.1/sqlparse/filters/tokens.py
D