Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/15623 )
Change subject: Add CentOS 8.1 support to bootstrap_system.sh ...................................................................... Patch Set 6: (5 comments) http://gerrit.cloudera.org:8080/#/c/15623/6//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/15623/6//COMMIT_MSG@14 PS6, Line 14: vrtual nit:virtual http://gerrit.cloudera.org:8080/#/c/15623/6//COMMIT_MSG@18 PS6, Line 18: curl is added to the list of required packages, required by : IMPALA-9149 delete this: curl was added for the no-NTP patch http://gerrit.cloudera.org:8080/#/c/15623/6//COMMIT_MSG@25 PS6, Line 25: necessry nit: necessary http://gerrit.cloudera.org:8080/#/c/15623/5/bin/bootstrap_system.sh File bin/bootstrap_system.sh: http://gerrit.cloudera.org:8080/#/c/15623/5/bin/bootstrap_system.sh@266 PS5, Line 266: sudo alternatives --add-slave python /usr/bin/python2 /usr/bin/pip pip /usr/bin/pip2 > I need to look at the other reviews, but there are very few places where we TL;DR: Impala (minus the shell) is still Python2 dependent, and Impala development requires serious commitment as far as system configuration is concerned. The Impala utility scripts (the ones not dependent on impala-python) should move to Python3 compatibility (I'll fiel a Jira if there's none yet). To me that looks like a separate, although important issue; until that's achieved, this option looks like a passable interim solution. @Tim: I agree that this is far from ideal for long-running environments. For better or worse, bootstrap_system.sh makes other changes as well, although probably none of them are as significant as this one if you assume a Python3 default before it is run. On the other hand Impala and Hadoop seems to be coupled to Python2 so much that it is hardly a surprise that Impala (as of now) assumes default Python to be Python2. @David: I've run a quick `git grep` for shebangs: laszlog@laszlog2:~/upstream/Impala$ git grep -l '^#!.* python' | wc -l 32 laszlog@laszlog2:~/upstream/Impala$ git grep -l '^#!.* python' -- shell | wc -l 13 IOW we have 32 scripts with "#!/usr/bin/env python", of which 13 files are located in the "shell" subtree. Most of the rest are utility scripts, like push_to_asf.py or the config generators like fe/src/test/resources/hive-site.xml.py; I assume these can be made Python3-ready (or tolerant) relatively easy. There are a few product scripts too, like common/function-registry/gen_builtins_catalog.py, or common/thrift/generate_error_codes.py, which may be more difficult, but I assume it would be more like a matter of time. Note that this doesn't consider any possible scripts outside the Impala tree (e.g. various private tooling or test frameworks or systems), neither does it consider scripts in other Hadoop components that might get invoked. I also saw that there are multiple existing Jira tickets about making various Python components compatible with Python3; I'll check those and file a new one for script updates if needed. http://gerrit.cloudera.org:8080/#/c/15623/5/bin/bootstrap_system.sh@367 PS5, Line 367: redhat8 indocker sudo -u postgres PGDATA=/var/lib/pgsql/data bash -c \ > Not a blocker, but it would be nice to have a way to avoid the duplication Yes, the support matrix grew large enough (redhat[6]/7/8, Ubuntu 14/16/18[/20 in the near future]) that some refactoring is called for. I'll file a Jira. -- To view, visit http://gerrit.cloudera.org:8080/15623 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3df5d48eca7a10219264e3604a4f05f072188e6e Gerrit-Change-Number: 15623 Gerrit-PatchSet: 6 Gerrit-Owner: Laszlo Gaal <laszlo.g...@cloudera.com> Gerrit-Reviewer: David Knupp <dkn...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Laszlo Gaal <laszlo.g...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Wed, 08 Apr 2020 16:49:45 +0000 Gerrit-HasComments: Yes
