Thank Zoltan and Laszlo for checking this!
For the git-ignore issue mentioned by Zoltan, I think we should remove
the line of "thirdparty" in .gitignore. All files in be/src/thirdparty
are tracked in the git index. In my local repo, if I add --no-index
option to git-check-ignore, I can see thirdparty is ignored as what we
saw in the release tarball:
$ git check-ignore -nv --no-index be/**/*
...
.gitignore:4:thirdparty be/src/thirdparty
...
In the release tarball, be/src/thirdparty is not in the git index
since there are no history. So they are ignored finally. I track the
history of why we have "thirdparty" in .gitignore. It's there in the
first commit of that file. But the purpose is not ignoring
be/src/thirdparty:
commit edc3a5518439ed59e6c46b04d0393018aabfa8c5
Author: carl <c...@cloudera.com>
Date: Fri Jul 29 15:49:39 2011 -0700
IMP-8. Update build scripts, etc., to reflect thirdparty/hadoop
and thirdparty/hive
...
create mode 100644 .gitignore
...
At that time we don't have be/src/thirdparty yet (It's created in
IMPALA-3223). I filed a JIRA to track this issue:
https://issues.apache.org/jira/browse/IMPALA-11201
--------------------------------------------------------------------
For the LZO build issue, I planned to fix it by this commit:
https://github.com/apache/impala/commit/eb1ed66fa435a722fa8c6a7c58ff53edc10c572e
It changes bootstrap_system.sh to clone the asf-3.4 branch of
Impala-lzo. However, the all-build-options-ub1604 job still fails.
Then I realized that it uses the cdh5-trunk branch by default. The
build script has codes like this:
IMPALA_LZO=${IMPALA_LZO:-}
if [[ -n "${IMPALA_LZO}" ]]; then
git init ~/Impala-lzo
pushd ~/Impala-lzo
GitWithRetry fetch https://github.com/cloudera/impala-lzo.git
IMPALA_LZO_BRANCH="${IMPALA_LZO_BRANCH:-cdh5-trunk}"
GitWithRetry fetch https://github.com/cloudera/impala-lzo.git
"${IMPALA_LZO_BRANCH}"
GitWithRetry checkout FETCH_HEAD
GitWithRetry rev-parse FETCH_HEAD
popd
...
By default IMPALA_LZO is not set. Thus the cdh5-trunk branch is used.
I think we don't need to be blocked by our Jenkins jobs, because users
are building Impala in their own envs. Do you think it's ok to go
ahead with RC3? https://github.com/apache/impala/tree/3.4.1-rc3
Thanks,
Quanlong
On Fri, Mar 25, 2022 at 12:21 AM Laszlo Gaal <laszlo.g...@cloudera.com> wrote:
>
> Thanks for jumping on it, Quanlong.
>
> > It's a pity that the logs don't print the commit hash. Maybe it's due
> > to some config differences.
>
> If you mean that the Jenkins run summary page doesn't display the git
> hash(es) of the project(s) they build:
> this happens because the current job configurations don't rely on Jenkins's
> built-in Git
> infrastructure to handle the operations. All (or at least all commonly
> used) jobs handle these on their own,
> in shell script fragments replicated throughout the various jobs.
>
> I'll make a pass through the jobs to let Jenkins handle the Git operations,
> but it will take me at least a few days to get to this.
>
> On another note, this problem also highlights the fact that the
> release-test job
> needs to know which impala-lzo branch to use. The job does not currently
> take this
> parameter, so it should be added, otherwise it will never be able to verify
> this release correctly.
>
> I'll let the community know when the fixes/enhancements land.
> In the meantime the suggested workaround for the release-test-ub1604 job is
> to check the results, and if the only failure is from the
> "all-build-options-ub1604" job then
> re-run just that job against the release candidate, passing in "asf-3.4"
> for the IMPALA_LZO_BRANCH parameter.
>
> Thank you for your patience,
>
> - LaszloG
>
>
>
> On Thu, Mar 24, 2022 at 10:18 AM Quanlong Huang <huangquanl...@gmail.com>
> wrote:
>
> > Sorry that there is a build issue on Impala-lzo. Thanks Laszlo Gaal
> > and Riza Suminto for reporting this to me offline!
> >
> > Impala-3.4 should depend on the asf-3.4 branch of Impala-lzo, instead
> > of the master branch. The build issue is revealed by the
> > all-build-options job:
> > https://jenkins.impala.io/job/all-build-options-ub1604/8069/
> >
> > I'm confused why my builds passed:
> > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/16037/
> > https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5440/
> >
> > It's a pity that the logs don't print the commit hash. Maybe it's due
> > to some config differences.
> >
> > Anyway, I'll post another RC soon. Thanks again for your votes and
> > sorry for the mistake!
> >
> > Quanlong
> >
> > On Thu, Mar 24, 2022 at 6:35 AM Joe McDonnell <joemcdonn...@cloudera.com>
> > wrote:
> > >
> > > +1 (binding)
> > >
> > > I verified the tarball contents, the signatures, the sha512 sum, built it
> > > from source, and verified the tests.
> > >
> > > On Wed, Mar 23, 2022 at 1:19 AM skyyws <sky...@163.com> wrote:
> > >
> > > > +1, thanks for Quanlong!
> > > >
> > > >
> > > >
> > > >
> > > > On 03/23/2022 15:28，Quanlong Huang<stigahu...@apache.org> wrote：
> > > > Hi all,
> > > >
> > > > This is a vote for Impala 3.4.1.
> > > >
> > > > The artifacts for testing can be downloaded from:
> > > > https://dist.apache.org/repos/dist/dev/impala/3.4.1/RC2/
> > > > Git tag: 3.4.1-rc2
> > https://github.com/apache/impala/releases/tag/3.4.1-rc2
> > > > Tree hash: 7358fb59d9c09558d732fe680311a833eb2966af
> > > >
> > > > Please vote +1 or -1. -1 votes should be accompanied by an explanation
> > > > of the reason. Only PMC members have binding votes, but other
> > > > community members are encouraged to cast non-binding votes. This vote
> > > > will pass if there are 3 binding +1 votes and more binding +1 votes
> > > > than -1 votes.
> > > >
> > > > This wiki page describes how to check the release before you vote:
> > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release#HowtoRelease-HowtoVoteonaReleaseCandidate
> > > >
> > > > I tested this by running exhaustive jobs for both with/without
> > LocalCatalog
> > > > mode, i.e. ubuntu-16.04-from-scratch and ubuntu-16.04-dockerised-tests.
> > > > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/16037/
> > > > https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5440/
> > > >
> > > > Note that this is RC2 since RC1 was blocked for having IMPALA-9242. We
> > > > will add back IMPALA-9242 in 3.4.2 after we fix IMPALA-10326. (Thank
> > > > Csaba for pointing out this!)
> > > >
> > > > Thanks,
> > > > Quanlong
> > > >
> >
