Abhishek Chennaka has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21873 )
Change subject: KUDU-3604: Pin Cython Version, Package Maintenance ...................................................................... KUDU-3604: Pin Cython Version, Package Maintenance Currently, users need to manually install Cython before installing the Kudu Python package. By running '$ pip install Cython' on most relatively new systems, they obtain Cython 3.x. However, our setup.py and build process are not compatible with Cython 3.x. In this patch, I've added a prerequisite step to setup.py that identifies and installs the correct Cython version before all the package build/install steps. Additionally, the Kudu Python package pulled from PyPi includes all the test files. This was resolved with a simple change in MANIFEST.in to exclude the test source folder. Moreover, end-users were also getting the test dependencies from setup.py because we had a single requirements.txt file for both distribution and development. I have now split the dependencies into two files: requirements.txt and requirements_dev.txt. The dependencies in requirements.txt are automatically pulled during the execution of setup.py. For development tasks, developers need to manually install requirements_dev.txt before running setup.py. The build-and-test.sh script was also updated to use requirements_dev.txt. Furthermore, there were several details and edge cases covered in the CI script related to older OS versions (e.g., el6) and unsupported Python versions (e.g., Python 2.6), as well as unpinned package versions. Since all packages are now version-pinned and these limitations are no longer relevant, I have cleaned up the package installation part in build-and-test.sh for the Python packages. As a follow-up patch, I plan to add more developer-facing documentation on how to set up the development environment, run unit tests, debug, etc. Change-Id: I45ad45c56fc3f3ecf57a413edbd0866fccbd8f79 Reviewed-on: http://gerrit.cloudera.org:8080/21816 Reviewed-by: Abhishek Chennaka <achenn...@cloudera.com> Tested-by: Abhishek Chennaka <achenn...@cloudera.com> (cherry picked from commit 8d9b3574a12bb98b79e29375c76f98a1a48af5ad) Reviewed-on: http://gerrit.cloudera.org:8080/21873 Reviewed-by: Alexey Serbin <ale...@apache.org> --- M build-support/jenkins/build-and-test.sh M python/MANIFEST.in M python/README.md M python/requirements.txt A python/requirements_dev.txt M python/setup.py A python/setup_prerequisites.py 7 files changed, 94 insertions(+), 143 deletions(-) Approvals: Alexey Serbin: Looks good to me, approved Abhishek Chennaka: Verified -- To view, visit http://gerrit.cloudera.org:8080/21873 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.18.x Gerrit-MessageType: merged Gerrit-Change-Id: I45ad45c56fc3f3ecf57a413edbd0866fccbd8f79 Gerrit-Change-Number: 21873 Gerrit-PatchSet: 2 Gerrit-Owner: Abhishek Chennaka <achenn...@cloudera.com> Gerrit-Reviewer: Abhishek Chennaka <achenn...@cloudera.com> Gerrit-Reviewer: Alexey Serbin <ale...@apache.org> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Marton Greber <greber...@gmail.com>
