The packaging support is now merged to branch-3.4.2. While testing the packages, I found some issues due to differences between the master and the 3.4.2 branch. Here are additional patches that we need for branch-3.4.2:
http://gerrit.cloudera.org:8080/21259 IMPALA-10262: (Addendum) fix incorrect paths of shared libs in building the ... http://gerrit.cloudera.org:8080/21260 IMPALA-9845: Point Maven and Ant downloads to stable locations http://gerrit.cloudera.org:8080/21261 IMPALA-10052: Expose daemon health endpoint for statestore and catalog http://gerrit.cloudera.org:8080/21262 IMPALA-12288: Add BUILD_WITH_NO_TESTS option to remove test targets http://gerrit.cloudera.org:8080/21263 IMPALA-12362: (part-1/4) Refactor service management scripts. http://gerrit.cloudera.org:8080/21264 IMPALA-12362: (part-2/4) Optimize default configurations for packaging module. http://gerrit.cloudera.org:8080/21265 IMPALA-12979: Avoid using wildcard in CLASSPATH Thanks, Quanlong On Sun, Mar 10, 2024 at 8:49 PM Quanlong Huang <huangquanl...@gmail.com> wrote: > Thank Zihao! These are really helpful! > > I just backport them + IMPALA-9572 (to make IMPALA-10257 a clean > cherry-pick): > > - IMPALA-9952: Fix page index filtering for empty pages > - IMPALA-9572: Fix DCHECK in nested Parquet scanning > - IMPALA-10257: Relax check for page filtering > - IMPALA-10310: Fix couldn't skip rows in parquet file on NextRowGroup > - IMPALA-12565: Fix crash triggered by calling pmod() UDF > - IMPALA-5476: Fix catalogd restart brings stale metadata > > Have ran CORE tests before merging them: > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/19977/ > > Another patch I want to add for each release is the package support > (IMPALA-10262). It helps to build DEB/RPM packages to ease users to try > Impala. There are some non-trivial conflicts that I resolved. Uploaded the > patch for review: > > https://gerrit.cloudera.org/c/21129/ IMPALA-10057: Fix log spew by using > jars in the classpath > https://gerrit.cloudera.org/c/21130/ IMPALA-10262: RPM/DEB Packaging > Support > > Note that IMPALA-10057 is needed to reduce some conflicts. Looking for > reviewers for them. > > Thanks, > Quanlong > > On Thu, Mar 7, 2024 at 5:17 PM Zihao Ye <eyiz...@163.com> wrote: > >> Thanks to Quanlong for the work on this, it's very valuable for those who >> can't upgrade their Hive. >> >> >> We backported the following patches to our internal Impala 3.4.0 cluster >> and they have been running well in production. These patches are small and >> don't seem to be in the 3.4.2 git log right now, so I suggest considering >> including these fixes in the upcoming 3.4.2 release: >> >> >> - IMPALA-12565: Fix crash triggered by calling pmod() UDF. >> - IMPALA-10310: Fix inability to skip rows in a parquet file on >> NextRowGroup. >> - IMPALA-9952: Fix page index filtering for empty pages. >> - IMPALA-10257: Relax check for page filtering. >> - IMPALA-5476: Fix issue where a catalogd restart results in stale >> metadata. >> >> >> Thanks, >> Zihao >> >> On 2024/02/08 00:11:19 Quanlong Huang wrote: >> Hi all, >> >> I've created the 3.4.2 branch and submitted a patch for review to fix the >> build issue: https://gerrit.cloudera.org/c/20995/ >> >> Thanks, >> Quanlong >> >> On Mon, Jan 29, 2024 at 10:03 AM Guojing Feng <fg...@gmail.com> wrote: >> >> volunteer +1, is neccecary to maintain 3.x releases since its very >> expensive to update Hive2 to Hive3 for legacy systems. >> >> Quanlong Huang <hu...@gmail.com> 于2024年1月27日周六 06:59写道： >> >> Hi all, >> >> We still have users on the 3.x branch since they can't upgrade their Hive >> to newer versions. It's reported that the 3.4.1 release is not compilable >> now (IMPALA-12755). It'd be nice to have a buildable release for 3.4. >> >> I propose doing a maintenance release of 3.4.2 to fix the build issue >> and I >> volunteer to manage the release of it. It can branch out based on the >> HEAD >> of 3.4.1. We can also add some fixes (some are already in 3.x): >> >> - IMPALA-11296: Fix infinite loop when reading orc files >> - IMPALA-11444: Fix wrong results when reading wide rows from ORC >> - IMPALA-12102: Avoid memory leaks in the handling of JNI exceptions >> - IMPALA-11406: Fix incorrect duration log for authorization >> >> Optional items: >> >> - Backport RPM/DEB packaging support (IMPALA-10262) so it's easier to >> distribute Impala. >> - Backport script changes to support building on Ubuntu 18.04 and >> 20.04 >> (IMPALA-10007). >> >> I tried a branch like this: >> https://github.com/stiga-huang/impala/commits/branch-3.4.2 >> It passed the CORE tests: >> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/19973/ >> >> Please speak up and let the community know your ideas, especially any bug >> fixes that you want to backport to 3.4.2. All feedback is welcome! >> >> PS This is not a VOTE thread but only a DISCUSS thread. >> >> Thanks, >> Quanlong >> >> >> >> >> >> >>
