Thank you for your help, Philip! >I added "IMPALA_LZO_BRANCH=2.x" to the properties file generated in cherrypick-2.x-and-test....
This fixes my second failure. For my first failure, it still exists for my recent run: https://jenkins.impala.io/job/cherrypick-2.x-and-test/641 -> https://jenkins.impala.io/job/parallel-all-tests/4992 -> https://jenkins.impala.io/job/ubuntu-16.04-build-only/5395 The ubuntu-16.04-build-only job doesn't check out to the correct branch of Impala-lzo. I think we need to add these in the Jenkins command before buildall.sh: (pushd $IMPALA_LZO && git checkout $IMPALA_LZO_BRANCH) We also need to pass IMPALA_LZO_BRANCH to this job. Do you think so? Could you help to have a try? Thanks, Quanlong On Thu, Feb 14, 2019 at 6:51 AM Philip Zeyliger <[email protected]> wrote: > Sorry about that. Some Cloudera-internal automation force-pushed that > branch. It's been fixed now, I hope. > > On Wed, Feb 13, 2019 at 2:00 PM Andrew Sherman <[email protected]> > wrote: > > > I am debugging some weird build failures which seem to be confused about > > branches in https://github.com/cloudera/impala-lzo.git > > 'master' has the latest changes, but 'origin/master', the default branch > > when you clone, points to some really old commit. > > > > Does anyone understand this? > > > > -Andrew > > > > asherman@asherman-desktop:~/git/asf/Impala-lzo$ git log -1 > --pretty=fuller > > master > > commit dccb1be88a5e237b06ae69cd99b048a38d9f024b (HEAD -> master) > > Author: Philip Zeyliger <[email protected]> > > AuthorDate: Tue Jan 22 14:21:42 2019 -0800 > > Commit: Philip Zeyliger <[email protected]> > > CommitDate: Fri Feb 1 11:04:06 2019 -0800 > > > > Adapt to new interface for AddDiskIoRange (IMPALA-7980) > > > > The change for IMPALA-7980 moves out the "are we done > > adding disk io ranges" accounting from AddDiskIoRange() > > to a new UpdateRemainingScanRangeSubmissions call. > > > > asherman@asherman-desktop:~/git/asf/Impala-lzo$ git log -1 > --pretty=fuller > > origin/master > > commit 9543908ec82245878c4060dec64a1533d7adbee6 > > (origin/master-backup-2019-01-23, origin/master) > > Author: Alex Behm <[email protected]> > > AuthorDate: Thu Dec 4 16:53:06 2014 -0800 > > Commit: Alex Behm <[email protected]> > > CommitDate: Fri Dec 5 14:28:25 2014 -0800 > > > > Remove data compaction flag. > > > > Change-Id: Ife72c7ee630b6ecd9024a3d33b7673b56fcf1fba > > > > I am tracking this at https://issues.apache.org/jira/browse/IMPALA-8200 > > > > > > On Wed, Feb 13, 2019 at 9:27 AM Philip Zeyliger <[email protected]> > > wrote: > > > > > I added "IMPALA_LZO_BRANCH=2.x" to the properties file generated in > > > cherrypick-2.x-and-test.... > > > > > > [image: image.png] > > > > > > On Tue, Feb 12, 2019 at 11:36 PM Quanlong Huang < > [email protected] > > > > > > wrote: > > > > > >> Thank Tim and Fredy's help! > > >> > > >> However, I still find some downstream Jenkins jobs not using the same > > >> IMPALA_LZO_BRANCH value. There're two failures due to this: > > >> > > >> 1) I ran gerrit-verify-dryrun for patch set 12429 > > >> with IMPALA_LZO_BRANCH=2.x. The job is > > >> https://jenkins.impala.io/job/gerrit-verify-dryrun/3760 > > >> This downstream job ubuntu-16.04-build-only still used master branch > of > > >> Impala-lzo thus failed: > > >> - ubuntu-16.04-build-only #5336 > > >> https://jenkins.impala.io/job/ubuntu-16.04-build-only/5336/ > > >> > > >> Looks like this job does not use IMPALA_LZO_BRANCH so not switch > branch > > of > > >> Impala-lzo. I think we should set IMPALA_LZO_BRANCH in > impala-config.sh > > >> and > > >> use it in buildall.sh. > > >> > > >> 2) I ran cherrypick-2.x-and-test. The job is > > >> https://jenkins.impala.io/job/cherrypick-2.x-and-test/638/ > > >> The problematic downstream jobs: > > >> - ubuntu-16.04-from-scratch #4470 > > >> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/4470/ > > >> - ubuntu-16.04-from-scratch #4471 > > >> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/4471/ > > >> > > >> Looks like the IMPALA_LZO_BRANCH is not set so they still use the > > default > > >> value (master). > > >> I don't have the permission to modify the job configs. Could anyone > help > > >> to > > >> have a look? > > >> > > >> Thanks, > > >> Quanlong > > >> > > >> On Mon, Feb 11, 2019 at 1:32 PM Quanlong Huang < > [email protected] > > > > > >> wrote: > > >> > > >> > Then branch-2.x needs this again: > > https://gerrit.cloudera.org/c/12339/. > > >> > Could anyone have a look at this? I encountered a build error at > > >> > https://gerrit.cloudera.org/c/12429/ due to this. > > >> > > > >> > Thanks, > > >> > Quanlong > > >> > > > >> > On Wed, Feb 6, 2019 at 1:15 AM Philip Zeyliger <[email protected] > > > > >> > wrote: > > >> > > > >> >> Thanks! (And Jim suggested the same thing.) > > >> >> > > >> >> My apologies for the churn. What I didn't realize is that > > >> >> https://jenkins.impala.io/job/parallel-all-tests/build takes a > > >> >> "IMPALA_LZO_BRANCH" parameter. I had foolishly assumed that the > > branch > > >> >> logic was in > > >> >> > > >> >> > > >> > > > https://gitbox.apache.org/repos/asf?p=impala.git;a=blob;f=bin/bootstrap_system.sh;h=cb869ffe79f3c35cf45b2f962a472274319d746c;hb=HEAD#l378 > > >> >> , > > >> >> but it's actually passed through in the Jenkins job. Yesterday Lars > > >> and I > > >> >> piped some of the stuff through to > > >> >> > https://jenkins.impala.io/job/gerrit-verify-dryrun/build?delay=0sec > > as > > >> >> well. > > >> >> > > >> >> Again, apologies for causing the churn. I should have worked it out > > in > > >> >> more > > >> >> detail. Please continue to let me know if things are still broken. > > >> >> > > >> >> -- Philip > > >> >> > > >> >> On Tue, Feb 5, 2019 at 1:00 AM Tim Armstrong < > > [email protected]> > > >> >> wrote: > > >> >> > > >> >> > I just changed the default to master, because that would make > > sense. > > >> >> > > > >> >> > On Tue, Feb 5, 2019 at 6:56 AM Quanlong Huang < > > >> [email protected]> > > >> >> > wrote: > > >> >> > > > >> >> > > Oh, I thought the default is master. Just abandoned my patch > > since > > >> >> it's > > >> >> > > redundant. There's a relative patch for impala-3.x to use the > > >> master > > >> >> > > branch: > > >> >> > > > > >> >> > > > > >> >> > > > >> >> > > >> > > > https://github.com/apache/impala/commit/365e35a36f142532044d8a34cbb87646bf9cb240 > > >> >> > > > > >> >> > > On Sun, Feb 3, 2019 at 4:40 AM Jim Apple <[email protected] > > > > >> >> wrote: > > >> >> > > > > >> >> > > > Do you want to change default branch to master, rather than > > >> >> cdh5-trunk? > > >> >> > > > > > >> >> > > > https://help.github.com/articles/setting-the-default-branch/ > > >> >> > > > > > >> >> > > > On Fri, Feb 1, 2019 at 3:51 PM Philip Zeyliger < > > >> [email protected] > > >> >> > > > >> >> > > > wrote: > > >> >> > > > > > >> >> > > > > "cdh5-trunk" was that branch, but I also pushed it to 2.x, > > for > > >> a > > >> >> bit > > >> >> > > more > > >> >> > > > > consistency. > > >> >> > > > > > > >> >> > > > > -- Philip > > >> >> > > > > > > >> >> > > > > On Fri, Feb 1, 2019 at 3:44 PM Quanlong Huang < > > >> >> > [email protected] > > >> >> > > > > > >> >> > > > > wrote: > > >> >> > > > > > > >> >> > > > > > Hi Philip, > > >> >> > > > > > > > >> >> > > > > > Thanks for your kindly reminder! Could you help to > create a > > >> tag > > >> >> in > > >> >> > > the > > >> >> > > > > > cloudera/Impala-lzo project to point to the previous > > commit? > > >> >> Then > > >> >> > > > > > it's convenient for branch-2.x to point to a pined > version. > > >> >> > > > > > > > >> >> > > > > > Thanks, > > >> >> > > > > > Quanlong > > >> >> > > > > > > > >> >> > > > > > On Sat, Feb 2, 2019 at 5:33 AM Philip Zeyliger < > > >> >> > [email protected]> > > >> >> > > > > > wrote: > > >> >> > > > > > > > >> >> > > > > > > Hi Fredy, > > >> >> > > > > > > > > >> >> > > > > > > I think you need to rebase your change onto mine. I've > > just > > >> >> done > > >> >> > so > > >> >> > > > by > > >> >> > > > > > > clicking the button, and we'll see. > > >> >> > > > > > > > > >> >> > > > > > > -- Philip > > >> >> > > > > > > > > >> >> > > > > > > On Fri, Feb 1, 2019 at 12:24 PM Fredy Wijaya < > > >> >> > [email protected] > > >> >> > > > > > >> >> > > > > > wrote: > > >> >> > > > > > > > > >> >> > > > > > > > gerrit-code-review-checks seems a bit unhappy with > the > > >> >> change, > > >> >> > > e.g. > > >> >> > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > >> >> > > > > > > > >> >> > > > > > > >> >> > > > > > >> >> > > > > >> >> > > > >> >> > > >> > > > https://jenkins.impala.io/job/gerrit-code-review-checks/1963/artifact/https%3A%5E%5Ejenkins.impala.io%5Ejob%5Eclang-tidy-ub1604%5E5041%5E/*view*/ > > >> >> > > > > > > > > > >> >> > > > > > > > + ./buildall.sh -skiptests -tidy -so -noclean > > >> >> > > > > > > > + echo 'tidy build failed; See > /home/ubuntu/tidylog.txt > > >> for > > >> >> > full > > >> >> > > > tidy > > >> >> > > > > > > > build output. Guess:' > > >> >> > > > > > > > tidy build failed; See /home/ubuntu/tidylog.txt for > > full > > >> >> tidy > > >> >> > > build > > >> >> > > > > > > > output. Guess: > > >> >> > > > > > > > + grep ': error: ' /home/ubuntu/tidylog.txt > > >> >> > > > > > > > > > /home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:140:17: > > >> >> error: > > >> >> > > > > > > > ‘class impala::HdfsScanNodeBase’ has no member > > named > > >> >> > > > > > > > ‘UpdateRemainingScanRangeSubmissions’ > > >> >> > > > > > > > > > /home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:168:14: > > >> >> error: > > >> >> > > > > > > > ‘class impala::HdfsScanNodeBase’ has no member > > named > > >> >> > > > > > > > ‘UpdateRemainingScanRangeSubmissions’ > > >> >> > > > > > > > > > /home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:169:59: > > >> >> error: > > >> >> > > no > > >> >> > > > > > > > matching function for call to > > >> >> > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > >> >> > > > > > > > >> >> > > > > > > >> >> > > > > > >> >> > > > > >> >> > > > >> >> > > >> > > > ‘impala::HdfsScanNodeBase::AddDiskIoRanges(std::vector<impala::io::ScanRange*>&)’ > > >> >> > > > > > > > > > /home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:200:49: > > >> >> error: > > >> >> > > no > > >> >> > > > > > > > matching function for call to > > >> >> > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > >> >> > > > > > > > >> >> > > > > > > >> >> > > > > > >> >> > > > > >> >> > > > >> >> > > >> > > > ‘impala::HdfsScanNodeBase::AddDiskIoRanges(std::vector<impala::io::ScanRange*>)’ > > >> >> > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > On Fri, Feb 1, 2019 at 1:20 PM Philip Zeyliger < > > >> >> > > > [email protected]> > > >> >> > > > > > > > wrote: > > >> >> > > > > > > > > > >> >> > > > > > > > > Hi folks, > > >> >> > > > > > > > > > > >> >> > > > > > > > > I just pushed > > >> >> > > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > >> >> > > > > > > > >> >> > > > > > > >> >> > > > > > >> >> > > > > >> >> > > > >> >> > > >> > > > https://gitbox.apache.org/repos/asf?p=impala.git;a=commit;h=a8e30506aafef14646d95a56fb87cf7c28d259d6 > > >> >> > > > > > > > > and > > >> >> > > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > >> >> > > > > > > > >> >> > > > > > > >> >> > > > > > >> >> > > > > >> >> > > > >> >> > > >> > > > https://github.com/cloudera/impala-lzo/commit/dccb1be88a5e237b06ae69cd99b048a38d9f024b > > >> >> > > > > > > > > to tackle IMPALA-7980. If you're working on master, > > and > > >> >> you > > >> >> > > have > > >> >> > > > > > > > > ../Impala-lzo checked out, you'll need to pull that > > >> repo. > > >> >> > > There's > > >> >> > > > > > also > > >> >> > > > > > > a > > >> >> > > > > > > > > divergence now for Impala-lzo between branches, so > if > > >> >> you're > > >> >> > > > still > > >> >> > > > > > > > working > > >> >> > > > > > > > > on 2.x (say), you'll need to figure out how to > manage > > >> >> that. > > >> >> > > > > > > > > > > >> >> > > > > > > > > Please let me know if you run into issues; I'll be > > >> happy > > >> >> to > > >> >> > > help. > > >> >> > > > > > > > > > > >> >> > > > > > > > > -- Philip > > >> >> > > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > >> >> > > > > > > > >> >> > > > > > > >> >> > > > > > >> >> > > > > >> >> > > > >> >> > > >> > > > >> > > > > > >
