Hello Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23976
to look at the new patch set (#18).
Change subject: IMPALA-14770: Upgrade impala-python3 to 3.11.14
......................................................................
IMPALA-14770: Upgrade impala-python3 to 3.11.14
This upgrades the impala-python3 virtualenv to use Python 3.11.14.
Upgrading to Python 3.11 allows us to use the most recent version
of most packages. Python 3.11 delivered performance improvements
over previous versions. It seems to reduce the runtime of precommit
by about 15-20 minutes (~4:40 -> ~4:25).
This fixes code using removed library functions:
- XML's getiterator() was removed and code needs to use iter()
- thread's isAlive() was removed and code needs to use is_alive()
This fixes some pieces of code to avoid warnings:
- The "pipes" package is deprecated, so this changes code using
pipes.quote() to use shlex.quote() instead.
- This changes some regex patterns to use raw strings to avoid
warnings about invalid escape sequences.
This upgrades a variety of packages (some of them required, many of them
optional):
1. Upgrades setuptools to a version from about a month ago
2. Upgrades requests to the latest version
- Addresses some warnings that show for pytest
- Requests got stricter about certificates and server-cert.pem
is no longer considered valid.
- This regenerates the server-key.pem/server-key-password.pem/server-cert.pem
to add a subject alternative name of localhost.
3. Upgrades python-magic to the latest and fixes dump_breakpad_symbols.py
4. Upgrades psutil to the latest and fixes tests/common/impala_cluster.py to
handle the changes in how it represents zombie processes
5. Upgrades several other Python packages to more recent versions without
needing code changes (allpairspy, flake8, hdfs, kazoo, prometheus-client,
six, sasl, avro, distro, pytz, Cython). Removes unused pg8000 package.
There were issues upgrading some other packages, so this does not
upgrade them:
- Newer versions of sqlparse have a regression in split.split()
that makes it inoperable for us.
- Newer versions of pexpect would require adjustment in the
interactive shell tests.
- New versions of pytest are not difficult but require additional changes
that would be better suited to a separate commit
Testing:
- Ran precommit
- Ran exhaustive release jobs
Change-Id: I9383ebfb310a4ee30134e02b195c332ffbb6a7ec
---
M be/src/testutil/certificates-info.txt
M be/src/testutil/server-cert.pem
M be/src/testutil/server-key-password.pem
M be/src/testutil/server-key.pem
M bin/dump_breakpad_symbols.py
M bin/impala-config.sh
M infra/python/deps/requirements.txt
M infra/python/deps/setuptools-requirements.txt
M tests/common/custom_cluster_test_suite.py
M tests/common/impala_cluster.py
M tests/common/resource_pool_config.py
M tests/common/test_result_verifier.py
M tests/comparison/cluster.py
M tests/comparison/leopard/controller.py
M tests/custom_cluster/test_admission_controller.py
M tests/stress/query_retries_stress_runner.py
M tests/util/auto_scaler.py
17 files changed, 164 insertions(+), 137 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/76/23976/18
--
To view, visit http://gerrit.cloudera.org:8080/23976
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9383ebfb310a4ee30134e02b195c332ffbb6a7ec
Gerrit-Change-Number: 23976
Gerrit-PatchSet: 18
Gerrit-Owner: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
