Hi Ed, Thanks for the detailed review. I wanted to address some of your questions. Regarding JIRAs: for the last few releases, the activity on the JIRAs was minimal. I would create it and then after completing the implementation, I would close it. The community still has the option to weigh in on the code changes by reviewing the PRs but that hasn't happened as well. At this point, writing JIRAs just for myself doesn't help anybody. If the community is interested in suggesting new features, I would be happy to address them to the best of my abilities.
The typo in the wiki is fixed. Since it is not a part of the release, it is just a confluence page, I don't believe it affects the viability of this release. Regarding python dependencies: The suggestion on the version numbers for dependencies is valid. I will update the installation instructions webpage with the relevant information. The python executable itself not being in the path is a corner case for most systems but is easy to fix. We should create a JIRA to track it and get it fixed for the next release. Thanks, Orhan Kislal ________________________________ From: Ed Espino <esp...@apache.org> Sent: Tuesday, February 28, 2023 11:58 AM To: dev@madlib.apache.org <dev@madlib.apache.org> Subject: Re: Apache MADlib 1.21.0 Release (RC2) !! External Email Edward Espino Table of Contents _________________ 1. Apache MADlib Version v1.21.0 RC2 2. Observations .. 1. Reviewing Jira status - QUESTION .. 2. 1.21.0 docs Review - PASSED .. 3. MADlib 1.21.0 wiki - old release reference .. 4. PGP signature verification - PASSED .. 5. SHA512 checksum verification - PASSED .. 6. RELEASE_NOTES reviewed - QUESTION .. 7. Copyright review in NOTICE file - PASSED .. 8. Apache RAT (mvn apache-rat:check) - PASSED .. 9. Python dependencies - QUESTION/OBSERVATION .. 10. Operating System: Rocky Linux 8.7 (Green Obsidian) x86_64 - PASSED .. 11. Operating System: Ubuntu 22.04.2 LTS - PASSED .. 12. Test cannot find "python" - FAILURE 1 Apache MADlib Version v1.21.0 RC2 =================================== As I am not a member of PMC or a committer. Here are my observations of the release. 2 Observations ============== 2.1 Reviewing Jira status - QUESTION ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Only a single Jira is fixed in this release. Is this expected? <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FMADLIB-1506&data=05%7C01%7Cokislal%40vmware.com%7Cb5b835eef540432aa60308db196a0ddb%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638131715521745368%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=tGKhJB8Hf%2BRTtemUo%2FSgsbmI4y5VXGcuyKTc8VpRJP0%3D&reserved=0> v1.21.0 Release Jira info <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fprojects%2FMADLIB%2Fversions%2F12352104&data=05%7C01%7Cokislal%40vmware.com%7Cb5b835eef540432aa60308db196a0ddb%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638131715521901621%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6XEJWiGxFRIBJyll%2FNPr5IApQB5%2BQ3bezLYfqK09N%2Fg%3D&reserved=0> 1 Issues in version 1 Issues done 0 Issues in progress 0 Issues to do I am curious why the v1.20.0 Jira release list is not in a finalized state. <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fprojects%2FMADLIB%2Fversions%2F12352103&data=05%7C01%7Cokislal%40vmware.com%7Cb5b835eef540432aa60308db196a0ddb%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638131715521901621%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=fBz7AcPojrjorlxDlmqtUBABADYAt2apYO25txfEodY%3D&reserved=0> Is it possible some of these issues are actually fixed in the v1.21.0 release? 16 Issues in version 2 Issues done 0 Issues in progress 14 Issues to do 2.2 1.21.0 docs Review - PASSED ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmadlib.apache.org%2Fdocs%2Frc%2Findex.html&data=05%7C01%7Cokislal%40vmware.com%7Cb5b835eef540432aa60308db196a0ddb%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638131715521901621%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2H8fpJDILF3sKpnTqpJqxAcfieEzqAIi7%2FPpXJWLA1M%3D&reserved=0> 2.3 MADlib 1.21.0 wiki - old release reference ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FMADLIB%2FMADlib%2B1.21.0&data=05%7C01%7Cokislal%40vmware.com%7Cb5b835eef540432aa60308db196a0ddb%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638131715521901621%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=0CzUJ2EqRDZ2FfbYxv%2BOy%2FsGtlDCtotiZ3SP9Cp4n0Q%3D&reserved=0> This page reference v1.20.0 release: Release Notes MADlib v1.20.0: 2.4 PGP signature verification - PASSED ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --> Good signature from "Venkatesh Raghavan (G!) <raghava...@vmware.com>" [unknown] 2.5 SHA512 checksum verification - PASSED ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2.6 RELEASE_NOTES reviewed - QUESTION ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The v1.21.0 release notes indicate three New Features, two Improvements and three Bug Fixes are included. However, a single Jira (MADLIB-1506) is listed for the release. 2.7 Copyright review in NOTICE file - PASSED ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Apache MADlib Copyright 2016-2023 The Apache Software Foundation. ... 2.8 Apache RAT (mvn apache-rat:check) - PASSED ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [INFO] Building madlib 1.21.0 ... (elided verbose output) [INFO] 289 resources included (use -debug for more details) [INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 approved: 280 licence. 2.9 Python dependencies - QUESTION/OBSERVATION ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I was able to complete install-check and dev-check activities by installing the following Python modules. What exactly are the required components for testing and runtime use? Minimally, these should be referenced in the release notes. keras tensorflow==1.14 dill pandas hyperopt==0.2.5 xgboost rtree scikit-learn When building against the Greenplum 6.23.1 release I had to set the following to find the appropriate Python 2.7 shared libraries embedded within the release installation directory. export LIBRARY_PATH=/usr/local/greenplum-db-6.23.1/lib:/usr/local/greenplum-db-6.23.1/ext/python/lib 2.10 Operating System: Rocky Linux 8.7 (Green Obsidian) x86_64 - PASSED ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - VMware Greenplum 6.23.1 (GA) - PostgreSQL 12.14 (built from source) - PostgreSQL 11.19 (built from source) - Apache MADlib apache-madlib-1.21.0-src.tar.gz Source tarball - install-check passed - dev-check passed 2.11 Operating System: Ubuntu 22.04.2 LTS - PASSED ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - PostgreSQL 12.14 (built from source) - PostgreSQL 11.19 (built from source) - Apache MADlib apache-madlib-1.21.0-src.tar.gz Source tarball - install-check passed - dev-check passed 2.12 Test cannot find "python" - FAILURE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Python 2 on the system revealed a hardcoded legacy issue. On both Rocky 8 and Ubuntu 22 the Python 2 executable is not installed as /usr/bin/python. Thus the deep_learning madlib_keras_gpu_info test fails. The function get_gpu_info_from_tensorflow expects to find it via the executable name "python" which is not satisfied and the test fails. Here is the reference; < https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fmadlib%2Fblob%2Fmaster%2Fsrc%2Fports%2Fpostgres%2Fmodules%2Fdeep_learning%2Fmadlib_keras_gpu_info.py_in%23L67&data=05%7C01%7Cokislal%40vmware.com%7Cb5b835eef540432aa60308db196a0ddb%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638131715521901621%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=VURQWkI2b3kOlmpxkTe1NdRyXopP9bZNyJJQ7JQjA5M%3D&reserved=0 > I was able to workaround this by creating the following symbolic link. This is possible because I did not install Python 3 which might have relied upon /usr/bin/python. This needs some investigation and a minimal reference in the release notes. ln -s /usr/bin/python2 /usr/bin/python !! External Email: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender.
