Thank Laszlo! The release-test-job passes now for RC3: https://jenkins.impala.io/job/release-test-ub1604/51/ I will raise a vote for RC3 soon.
Thanks, Quanlong On Fri, Mar 25, 2022 at 9:46 PM Laszlo Gaal <laszlo.g...@cloudera.com> wrote: > > As for IMPALA_LZO and the Jenkins build script: > cdh5-trunk is an ancient branch of Impala-lzo, it supported > the 2.x branch of Impala. > The last commit on impala-lzo/cdh5-trunk is from 10/18/2018, which is > roughly the same timeframe when the community agreed to stop > active maintenance on the Impala 2.x branch; e.g. see Philip's mail at > https://lists.apache.org/thread/lghnszw8y2lyqr12zr477jhyy6m1jl7o > > My proposal (and offer) for the Jenkins build script is to change the > default > branch it clones to asf-3.4 per Quanlong's investigation above. > I'll change it later today unless someone objects. > > Thank you, > > - LaszloG > > On Fri, Mar 25, 2022 at 4:35 AM Quanlong Huang <huangquanl...@gmail.com> > wrote: > > > Thank Zoltan and Laszlo for checking this! > > > > For the git-ignore issue mentioned by Zoltan, I think we should remove > > the line of "thirdparty" in .gitignore. All files in be/src/thirdparty > > are tracked in the git index. In my local repo, if I add --no-index > > option to git-check-ignore, I can see thirdparty is ignored as what we > > saw in the release tarball: > > > > $ git check-ignore -nv --no-index be/**/* > > ... > > .gitignore:4:thirdparty be/src/thirdparty > > ... > > > > In the release tarball, be/src/thirdparty is not in the git index > > since there are no history. So they are ignored finally. I track the > > history of why we have "thirdparty" in .gitignore. It's there in the > > first commit of that file. But the purpose is not ignoring > > be/src/thirdparty: > > > > commit edc3a5518439ed59e6c46b04d0393018aabfa8c5 > > Author: carl <c...@cloudera.com> > > Date: Fri Jul 29 15:49:39 2011 -0700 > > IMP-8. Update build scripts, etc., to reflect thirdparty/hadoop > > and thirdparty/hive > > ... > > create mode 100644 .gitignore > > ... > > > > At that time we don't have be/src/thirdparty yet (It's created in > > IMPALA-3223). I filed a JIRA to track this issue: > > https://issues.apache.org/jira/browse/IMPALA-11201 > > > > -------------------------------------------------------------------- > > For the LZO build issue, I planned to fix it by this commit: > > > > https://github.com/apache/impala/commit/eb1ed66fa435a722fa8c6a7c58ff53edc10c572e > > It changes bootstrap_system.sh to clone the asf-3.4 branch of > > Impala-lzo. However, the all-build-options-ub1604 job still fails. > > Then I realized that it uses the cdh5-trunk branch by default. The > > build script has codes like this: > > > > IMPALA_LZO=${IMPALA_LZO:-} > > if [[ -n "${IMPALA_LZO}" ]]; then > > git init ~/Impala-lzo > > pushd ~/Impala-lzo > > GitWithRetry fetch https://github.com/cloudera/impala-lzo.git > > IMPALA_LZO_BRANCH="${IMPALA_LZO_BRANCH:-cdh5-trunk}" > > GitWithRetry fetch https://github.com/cloudera/impala-lzo.git > > "${IMPALA_LZO_BRANCH}" > > GitWithRetry checkout FETCH_HEAD > > GitWithRetry rev-parse FETCH_HEAD > > popd > > ... > > > > By default IMPALA_LZO is not set. Thus the cdh5-trunk branch is used. > > > > I think we don't need to be blocked by our Jenkins jobs, because users > > are building Impala in their own envs. Do you think it's ok to go > > ahead with RC3? https://github.com/apache/impala/tree/3.4.1-rc3 > > > > Thanks, > > Quanlong > > > > > > On Fri, Mar 25, 2022 at 12:21 AM Laszlo Gaal <laszlo.g...@cloudera.com> > > wrote: > > > > > > Thanks for jumping on it, Quanlong. > > > > > > > It's a pity that the logs don't print the commit hash. Maybe it's due > > > > to some config differences. > > > > > > If you mean that the Jenkins run summary page doesn't display the git > > > hash(es) of the project(s) they build: > > > this happens because the current job configurations don't rely on > > Jenkins's > > > built-in Git > > > infrastructure to handle the operations. All (or at least all commonly > > > used) jobs handle these on their own, > > > in shell script fragments replicated throughout the various jobs. > > > > > > I'll make a pass through the jobs to let Jenkins handle the Git > > operations, > > > but it will take me at least a few days to get to this. > > > > > > On another note, this problem also highlights the fact that the > > > release-test job > > > needs to know which impala-lzo branch to use. The job does not currently > > > take this > > > parameter, so it should be added, otherwise it will never be able to > > verify > > > this release correctly. > > > > > > I'll let the community know when the fixes/enhancements land. > > > In the meantime the suggested workaround for the release-test-ub1604 job > > is > > > to check the results, and if the only failure is from the > > > "all-build-options-ub1604" job then > > > re-run just that job against the release candidate, passing in "asf-3.4" > > > for the IMPALA_LZO_BRANCH parameter. > > > > > > Thank you for your patience, > > > > > > - LaszloG > > > > > > > > > > > > On Thu, Mar 24, 2022 at 10:18 AM Quanlong Huang <huangquanl...@gmail.com > > > > > > wrote: > > > > > > > Sorry that there is a build issue on Impala-lzo. Thanks Laszlo Gaal > > > > and Riza Suminto for reporting this to me offline! > > > > > > > > Impala-3.4 should depend on the asf-3.4 branch of Impala-lzo, instead > > > > of the master branch. The build issue is revealed by the > > > > all-build-options job: > > > > https://jenkins.impala.io/job/all-build-options-ub1604/8069/ > > > > > > > > I'm confused why my builds passed: > > > > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/16037/ > > > > https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5440/ > > > > > > > > It's a pity that the logs don't print the commit hash. Maybe it's due > > > > to some config differences. > > > > > > > > Anyway, I'll post another RC soon. Thanks again for your votes and > > > > sorry for the mistake! > > > > > > > > Quanlong > > > > > > > > On Thu, Mar 24, 2022 at 6:35 AM Joe McDonnell < > > joemcdonn...@cloudera.com> > > > > wrote: > > > > > > > > > > +1 (binding) > > > > > > > > > > I verified the tarball contents, the signatures, the sha512 sum, > > built it > > > > > from source, and verified the tests. > > > > > > > > > > On Wed, Mar 23, 2022 at 1:19 AM skyyws <sky...@163.com> wrote: > > > > > > > > > > > +1, thanks for Quanlong! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 03/23/2022 15:28，Quanlong Huang<stigahu...@apache.org> wrote： > > > > > > Hi all, > > > > > > > > > > > > This is a vote for Impala 3.4.1. > > > > > > > > > > > > The artifacts for testing can be downloaded from: > > > > > > https://dist.apache.org/repos/dist/dev/impala/3.4.1/RC2/ > > > > > > Git tag: 3.4.1-rc2 > > > > https://github.com/apache/impala/releases/tag/3.4.1-rc2 > > > > > > Tree hash: 7358fb59d9c09558d732fe680311a833eb2966af > > > > > > > > > > > > Please vote +1 or -1. -1 votes should be accompanied by an > > explanation > > > > > > of the reason. Only PMC members have binding votes, but other > > > > > > community members are encouraged to cast non-binding votes. This > > vote > > > > > > will pass if there are 3 binding +1 votes and more binding +1 votes > > > > > > than -1 votes. > > > > > > > > > > > > This wiki page describes how to check the release before you vote: > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release#HowtoRelease-HowtoVoteonaReleaseCandidate > > > > > > > > > > > > I tested this by running exhaustive jobs for both with/without > > > > LocalCatalog > > > > > > mode, i.e. ubuntu-16.04-from-scratch and > > ubuntu-16.04-dockerised-tests. > > > > > > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/16037/ > > > > > > https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5440/ > > > > > > > > > > > > Note that this is RC2 since RC1 was blocked for having > > IMPALA-9242. We > > > > > > will add back IMPALA-9242 in 3.4.2 after we fix IMPALA-10326. > > (Thank > > > > > > Csaba for pointing out this!) > > > > > > > > > > > > Thanks, > > > > > > Quanlong > > > > > > > > > > > >
