As for IMPALA_LZO and the Jenkins build script:
cdh5-trunk is an ancient branch of Impala-lzo, it supported
the 2.x branch of Impala.
The last commit on impala-lzo/cdh5-trunk is from 10/18/2018, which is
roughly the same timeframe when the community agreed to stop
active maintenance on the Impala 2.x branch; e.g. see Philip's mail at
https://lists.apache.org/thread/lghnszw8y2lyqr12zr477jhyy6m1jl7o

My proposal (and offer) for the Jenkins build script is to change the
default
branch it clones to asf-3.4 per Quanlong's investigation above.
I'll change it later today unless someone objects.

Thank you,

  - LaszloG

On Fri, Mar 25, 2022 at 4:35 AM Quanlong Huang <huangquanl...@gmail.com>
wrote:

> Thank Zoltan and Laszlo for checking this!
>
> For the git-ignore issue mentioned by Zoltan, I think we should remove
> the line of "thirdparty" in .gitignore. All files in be/src/thirdparty
> are tracked in the git index. In my local repo, if I add --no-index
> option to git-check-ignore, I can see thirdparty is ignored as what we
> saw in the release tarball:
>
> $ git check-ignore -nv --no-index be/**/*
> ...
> .gitignore:4:thirdparty be/src/thirdparty
> ...
>
> In the release tarball, be/src/thirdparty is not in the git index
> since there are no history. So they are ignored finally. I track the
> history of why we have "thirdparty" in .gitignore. It's there in the
> first commit of that file. But the purpose is not ignoring
> be/src/thirdparty:
>
> commit edc3a5518439ed59e6c46b04d0393018aabfa8c5
> Author: carl <c...@cloudera.com>
> Date:   Fri Jul 29 15:49:39 2011 -0700
>     IMP-8. Update build scripts, etc., to reflect thirdparty/hadoop
> and thirdparty/hive
> ...
>  create mode 100644 .gitignore
> ...
>
> At that time we don't have be/src/thirdparty yet (It's created in
> IMPALA-3223). I filed a JIRA to track this issue:
> https://issues.apache.org/jira/browse/IMPALA-11201
>
> --------------------------------------------------------------------
> For the LZO build issue, I planned to fix it by this commit:
>
> https://github.com/apache/impala/commit/eb1ed66fa435a722fa8c6a7c58ff53edc10c572e
> It changes bootstrap_system.sh to clone the asf-3.4 branch of
> Impala-lzo. However, the all-build-options-ub1604 job still fails.
> Then I realized that it uses the cdh5-trunk branch by default. The
> build script has codes like this:
>
> IMPALA_LZO=${IMPALA_LZO:-}
> if [[ -n "${IMPALA_LZO}" ]]; then
>   git init ~/Impala-lzo
>   pushd ~/Impala-lzo
>   GitWithRetry fetch https://github.com/cloudera/impala-lzo.git
>   IMPALA_LZO_BRANCH="${IMPALA_LZO_BRANCH:-cdh5-trunk}"
>   GitWithRetry fetch https://github.com/cloudera/impala-lzo.git
> "${IMPALA_LZO_BRANCH}"
>   GitWithRetry checkout FETCH_HEAD
>   GitWithRetry rev-parse FETCH_HEAD
>   popd
> ...
>
> By default IMPALA_LZO is not set. Thus the cdh5-trunk branch is used.
>
> I think we don't need to be blocked by our Jenkins jobs, because users
> are building Impala in their own envs. Do you think it's ok to go
> ahead with RC3? https://github.com/apache/impala/tree/3.4.1-rc3
>
> Thanks,
> Quanlong
>
>
> On Fri, Mar 25, 2022 at 12:21 AM Laszlo Gaal <laszlo.g...@cloudera.com>
> wrote:
> >
> > Thanks for jumping on it, Quanlong.
> >
> > > It's a pity that the logs don't print the commit hash. Maybe it's due
> > > to some config differences.
> >
> > If you mean that the Jenkins run summary page doesn't display the git
> > hash(es) of the project(s) they build:
> > this happens because the current job configurations don't rely on
> Jenkins's
> > built-in Git
> > infrastructure to handle the operations. All (or at least all commonly
> > used) jobs handle these on their own,
> > in shell script fragments replicated throughout the various jobs.
> >
> > I'll make a pass through the jobs to let Jenkins handle the Git
> operations,
> > but it will take me at least a few days to get to this.
> >
> > On another note, this problem also highlights the fact that the
> > release-test job
> > needs to know which impala-lzo branch to use. The job does not currently
> > take this
> > parameter, so it should be added, otherwise it will never be able to
> verify
> > this release correctly.
> >
> > I'll let the community know when the fixes/enhancements land.
> > In the meantime the suggested workaround for the release-test-ub1604 job
> is
> > to check the results, and if the only failure is from the
> > "all-build-options-ub1604" job then
> > re-run just that job against the release candidate, passing in "asf-3.4"
> > for the IMPALA_LZO_BRANCH parameter.
> >
> > Thank you for your patience,
> >
> >   - LaszloG
> >
> >
> >
> > On Thu, Mar 24, 2022 at 10:18 AM Quanlong Huang <huangquanl...@gmail.com
> >
> > wrote:
> >
> > > Sorry that there is a build issue on Impala-lzo. Thanks Laszlo Gaal
> > > and Riza Suminto for reporting this to me offline!
> > >
> > > Impala-3.4 should depend on the asf-3.4 branch of Impala-lzo, instead
> > > of the master branch. The build issue is revealed by the
> > > all-build-options job:
> > > https://jenkins.impala.io/job/all-build-options-ub1604/8069/
> > >
> > > I'm confused why my builds passed:
> > > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/16037/
> > > https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5440/
> > >
> > > It's a pity that the logs don't print the commit hash. Maybe it's due
> > > to some config differences.
> > >
> > > Anyway, I'll post another RC soon. Thanks again for your votes and
> > > sorry for the mistake!
> > >
> > > Quanlong
> > >
> > > On Thu, Mar 24, 2022 at 6:35 AM Joe McDonnell <
> joemcdonn...@cloudera.com>
> > > wrote:
> > > >
> > > > +1 (binding)
> > > >
> > > > I verified the tarball contents, the signatures, the sha512 sum,
> built it
> > > > from source, and verified the tests.
> > > >
> > > > On Wed, Mar 23, 2022 at 1:19 AM skyyws <sky...@163.com> wrote:
> > > >
> > > > > +1, thanks for Quanlong!
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On 03/23/2022 15:28,Quanlong Huang<stigahu...@apache.org> wrote:
> > > > > Hi all,
> > > > >
> > > > > This is a vote for Impala 3.4.1.
> > > > >
> > > > > The artifacts for testing can be downloaded from:
> > > > > https://dist.apache.org/repos/dist/dev/impala/3.4.1/RC2/
> > > > > Git tag: 3.4.1-rc2
> > > https://github.com/apache/impala/releases/tag/3.4.1-rc2
> > > > > Tree hash: 7358fb59d9c09558d732fe680311a833eb2966af
> > > > >
> > > > > Please vote +1 or -1. -1 votes should be accompanied by an
> explanation
> > > > > of the reason. Only PMC members have binding votes, but other
> > > > > community members are encouraged to cast non-binding votes. This
> vote
> > > > > will pass if there are 3 binding +1 votes and more binding +1 votes
> > > > > than -1 votes.
> > > > >
> > > > > This wiki page describes how to check the release before you vote:
> > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release#HowtoRelease-HowtoVoteonaReleaseCandidate
> > > > >
> > > > > I tested this by running exhaustive jobs for both with/without
> > > LocalCatalog
> > > > > mode, i.e. ubuntu-16.04-from-scratch and
> ubuntu-16.04-dockerised-tests.
> > > > > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/16037/
> > > > > https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5440/
> > > > >
> > > > > Note that this is RC2 since RC1 was blocked for having
> IMPALA-9242. We
> > > > > will add back IMPALA-9242 in 3.4.2 after we fix IMPALA-10326.
> (Thank
> > > > > Csaba for pointing out this!)
> > > > >
> > > > > Thanks,
> > > > > Quanlong
> > > > >
> > >
>

Reply via email to