[jira] [Resolved] (HDDS-1548) Jenkins precommit build is broken for Ozone
[ https://issues.apache.org/jira/browse/HDDS-1548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang resolved HDDS-1548. - Resolution: Fixed Jenkins Precommit build updated to reflect the changes. > Jenkins precommit build is broken for Ozone > --- > > Key: HDDS-1548 > URL: https://issues.apache.org/jira/browse/HDDS-1548 > Project: Hadoop Distributed Data Store > Issue Type: Improvement >Reporter: Eric Yang >Assignee: Eric Yang > Priority: Blocker > > HDDS Jenkins precommit build has been broken since Build 2685 Date May 13, > 2019 11:00:40PM. It looks like the precommit build depends on Yetus trunk. > This is extremely risky when Yetus trunk breaks, it also breaks precommit > build for Ozone. Precommit build must use a released version of Yetus to > prevent cascaded regression. > A second problem is the precommit build also depends on Marton's own personal > website to download ozone.sh. It would be best to version control ozone.sh > in hadoop-ozone/dev-support directory to prevent unpredictable changes to > ozone.sh at different time, which can make precommit build report > indeterministic. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDDS-1548) Jenkins precommit build is broken for Ozone
Eric Yang created HDDS-1548: --- Summary: Jenkins precommit build is broken for Ozone Key: HDDS-1548 URL: https://issues.apache.org/jira/browse/HDDS-1548 Project: Hadoop Distributed Data Store Issue Type: Improvement Reporter: Eric Yang HDDS Jenkins precommit build has been broken since Build 2685 Date May 13, 2019 11:00:40PM. It looks like the precommit build depends on Yetus trunk. This is extremely risky when Yetus trunk breaks, it also breaks precommit build for Ozone. Precommit build must use a released version of Yetus to prevent cascaded regression. A second problem is the precommit build also depends on Marton's own personal website to download ozone.sh. It would be best to version control ozone.sh in hadoop-ozone/dev-support directory to prevent unpredictable changes to ozone.sh at different time, which can make precommit build report indeterministic. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Re: Jenkins precommit build for HDFS failing
Solved by https://issues.apache.org/jira/browse/INFRA-17068. 张铎(Duo Zhang) 于2018年9月28日周五 上午9:37写道： > I tried to upgrade the yetus version for hbase pre commit job and the > error message is the same with hadoop then. And the problem is that when > fetching the patch we get a 4xx, so we just apply a nonexistent file... > > jira_http_fetch: https://issues.apache.org/jira/browse/HBASE-21233 returned > 4xx status code. Maybe incorrect username/password? > > > Ted Yu 于2018年9月28日周五 上午6:20写道： > >> Over in hbase precommit, I saw this: >> https://builds.apache.org/job/PreCommit-HBASE-Build/14514/console >> >> Resolving deltas: 86% (114758/133146), completed with 87 local >> objects.*21:06:47* fatal: pack has 18388 unresolved deltas*21:06:47* >> fatal: index-pack failed*21:06:47* *21:06:47* at >> >> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)*21:06:47* >> at >> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)*21:06:47* >> at >> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72) >> >> >> It seems QA machine(s) may have trouble accessing git. >> >> >> I wonder if the 'index-pack failed' error would lead to patch not >> being recognized. >> >> >> FYI >> >> >> On Thu, Sep 27, 2018 at 3:02 PM Ajay Kumar >> wrote: >> >> > Hi, >> > >> > Jenkins precommit build for HDFS is failing with error that patch >> doesn’t >> > apply to trunk, even when patch applies to trunk. >> > I see other build failures with same error for other patches as well. >> > Wanted to reach out to know if it is a known issue. >> > >> > >> > Vote >> > >> > Subsystem >> > >> > Runtime >> > >> > Comment >> > >> > 0 >> > >> > reexec >> > >> > 0m 0s >> > >> > Docker mode activated. >> > >> > -1 >> > >> > patch >> > >> > 0m 5s >> > >> > HDFS-13941 does not apply to trunk. Rebase required? Wrong Branch? See >> > https://wiki.apache.org/hadoop/HowToContribute for help. >> > >> > >> > Subsystem >> > >> > Report/Notes >> > >> > JIRA Issue >> > >> > HDFS-13941 >> > >> > Console output >> > >> > https://builds.apache.org/job/PreCommit-HDFS-Build/25154/console >> > >> > Powered by >> > >> > Apache Yetus 0.8.0 http://yetus.apache.org >> > >> > >> > >> > -1 overall >> > >> > >> > Vote >> > >> > Subsystem >> > >> > Runtime >> > >> > Comment >> > >> > 0 >> > >> > reexec >> > >> > 0m 0s >> > >> > Docker mode activated. >> > >> > -1 >> > >> > patch >> > >> > 0m 4s >> > >> > HDFS-13877 does not apply to trunk. Rebase required? Wrong Branch? See >> > https://wiki.apache.org/hadoop/HowToContribute for help. >> > >> > >> > Subsystem >> > >> > Report/Notes >> > >> > JIRA Issue >> > >> > HDFS-13877 >> > >> > Console output >> > >> > https://builds.apache.org/job/PreCommit-HDFS-Build/25155/console >> > >> > Powered by >> > >> > Apache Yetus 0.8.0 http://yetus.apache.org >> > >> > >> > >> > >> > >> > Thanks, >> > Ajay Kumar >> > >> >
Re: Jenkins precommit build for HDFS failing
I tried to upgrade the yetus version for hbase pre commit job and the error message is the same with hadoop then. And the problem is that when fetching the patch we get a 4xx, so we just apply a nonexistent file... jira_http_fetch: https://issues.apache.org/jira/browse/HBASE-21233 returned 4xx status code. Maybe incorrect username/password? Ted Yu 于2018年9月28日周五 上午6:20写道： > Over in hbase precommit, I saw this: > https://builds.apache.org/job/PreCommit-HBASE-Build/14514/console > > Resolving deltas: 86% (114758/133146), completed with 87 local > objects.*21:06:47* fatal: pack has 18388 unresolved deltas*21:06:47* > fatal: index-pack failed*21:06:47* *21:06:47* at > > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)*21:06:47* > at > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)*21:06:47* > at > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72) > > > It seems QA machine(s) may have trouble accessing git. > > > I wonder if the 'index-pack failed' error would lead to patch not > being recognized. > > > FYI > > > On Thu, Sep 27, 2018 at 3:02 PM Ajay Kumar > wrote: > > > Hi, > > > > Jenkins precommit build for HDFS is failing with error that patch doesn’t > > apply to trunk, even when patch applies to trunk. > > I see other build failures with same error for other patches as well. > > Wanted to reach out to know if it is a known issue. > > > > > > Vote > > > > Subsystem > > > > Runtime > > > > Comment > > > > 0 > > > > reexec > > > > 0m 0s > > > > Docker mode activated. > > > > -1 > > > > patch > > > > 0m 5s > > > > HDFS-13941 does not apply to trunk. Rebase required? Wrong Branch? See > > https://wiki.apache.org/hadoop/HowToContribute for help. > > > > > > Subsystem > > > > Report/Notes > > > > JIRA Issue > > > > HDFS-13941 > > > > Console output > > > > https://builds.apache.org/job/PreCommit-HDFS-Build/25154/console > > > > Powered by > > > > Apache Yetus 0.8.0 http://yetus.apache.org > > > > > > > > -1 overall > > > > > > Vote > > > > Subsystem > > > > Runtime > > > > Comment > > > > 0 > > > > reexec > > > > 0m 0s > > > > Docker mode activated. > > > > -1 > > > > patch > > > > 0m 4s > > > > HDFS-13877 does not apply to trunk. Rebase required? Wrong Branch? See > > https://wiki.apache.org/hadoop/HowToContribute for help. > > > > > > Subsystem > > > > Report/Notes > > > > JIRA Issue > > > > HDFS-13877 > > > > Console output > > > > https://builds.apache.org/job/PreCommit-HDFS-Build/25155/console > > > > Powered by > > > > Apache Yetus 0.8.0 http://yetus.apache.org > > > > > > > > > > > > Thanks, > > Ajay Kumar > > >
Re: Jenkins precommit build for HDFS failing
Over in hbase precommit, I saw this: https://builds.apache.org/job/PreCommit-HBASE-Build/14514/console Resolving deltas: 86% (114758/133146), completed with 87 local objects.*21:06:47* fatal: pack has 18388 unresolved deltas*21:06:47* fatal: index-pack failed*21:06:47* *21:06:47* at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2002)*21:06:47* at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1721)*21:06:47* at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72) It seems QA machine(s) may have trouble accessing git. I wonder if the 'index-pack failed' error would lead to patch not being recognized. FYI On Thu, Sep 27, 2018 at 3:02 PM Ajay Kumar wrote: > Hi, > > Jenkins precommit build for HDFS is failing with error that patch doesn’t > apply to trunk, even when patch applies to trunk. > I see other build failures with same error for other patches as well. > Wanted to reach out to know if it is a known issue. > > > Vote > > Subsystem > > Runtime > > Comment > > 0 > > reexec > > 0m 0s > > Docker mode activated. > > -1 > > patch > > 0m 5s > > HDFS-13941 does not apply to trunk. Rebase required? Wrong Branch? See > https://wiki.apache.org/hadoop/HowToContribute for help. > > > Subsystem > > Report/Notes > > JIRA Issue > > HDFS-13941 > > Console output > > https://builds.apache.org/job/PreCommit-HDFS-Build/25154/console > > Powered by > > Apache Yetus 0.8.0 http://yetus.apache.org > > > > -1 overall > > > Vote > > Subsystem > > Runtime > > Comment > > 0 > > reexec > > 0m 0s > > Docker mode activated. > > -1 > > patch > > 0m 4s > > HDFS-13877 does not apply to trunk. Rebase required? Wrong Branch? See > https://wiki.apache.org/hadoop/HowToContribute for help. > > > Subsystem > > Report/Notes > > JIRA Issue > > HDFS-13877 > > Console output > > https://builds.apache.org/job/PreCommit-HDFS-Build/25155/console > > Powered by > > Apache Yetus 0.8.0 http://yetus.apache.org > > > > > > Thanks, > Ajay Kumar >
Jenkins precommit build for HDFS failing
Hi, Jenkins precommit build for HDFS is failing with error that patch doesn’t apply to trunk, even when patch applies to trunk. I see other build failures with same error for other patches as well. Wanted to reach out to know if it is a known issue. Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 5s HDFS-13941 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HDFS-13941 Console output https://builds.apache.org/job/PreCommit-HDFS-Build/25154/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 4s HDFS-13877 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HDFS-13877 Console output https://builds.apache.org/job/PreCommit-HDFS-Build/25155/console Powered by Apache Yetus 0.8.0 http://yetus.apache.org Thanks, Ajay Kumar
Re: Jenkins precommit-*-build
Thank you Allen! --Konst On Tue, May 5, 2015 at 5:14 PM, Allen Wittenauer wrote: > > HDFS, MAPREDUCE, and YARN have been migrated. > > Let me know of any issues and I’ll try to get to them as I can. This > should be the end of the Jenkins race conditions for our pre commits! > *crosses fingers* > > > >
Re: Jenkins precommit-*-build
Thanks, Allen. This has long been a thorn in our side, and it's really good to see someone work on it. cheers, Colin On Tue, May 5, 2015 at 2:59 PM, Allen Wittenauer wrote: > TL;DR: > > Heads up: I’m going to hack on these scripts to fix the race > conditions. > > > > Presently, we have a pretty wicked race condition in the precommit > scripts, namely writing to a directory that is outside of Jenkins' workspace > directory. I’ve committed changes this morning that should allow us to run > test-patch.sh inside the source directory without triggering rat checks, will > survive a git clean, etc. so long as we use a Well Known Directory for > test-patch.sh’s tmp space. That’s one half of the race condition. The other > half is that now Jenkins needs to be configured to use this directory. That > requires changes in Jenkins. > > I’ll be making the changes to the HADOOP precommit here in a bit. > I’ll kick off some runs and watch for any issues. If not, I’ll be syncing > the HDFS, YARN, & MR pre commit scripts. (right now, there are slight > differences between them. Hooray!) > > Thanks.
Re: Jenkins precommit-*-build
HDFS, MAPREDUCE, and YARN have been migrated. Let me know of any issues and I’ll try to get to them as I can. This should be the end of the Jenkins race conditions for our pre commits! *crosses fingers*
Jenkins precommit-*-build
TL;DR: Heads up: I’m going to hack on these scripts to fix the race conditions. Presently, we have a pretty wicked race condition in the precommit scripts, namely writing to a directory that is outside of Jenkins' workspace directory. I’ve committed changes this morning that should allow us to run test-patch.sh inside the source directory without triggering rat checks, will survive a git clean, etc. so long as we use a Well Known Directory for test-patch.sh’s tmp space. That’s one half of the race condition. The other half is that now Jenkins needs to be configured to use this directory. That requires changes in Jenkins. I’ll be making the changes to the HADOOP precommit here in a bit. I’ll kick off some runs and watch for any issues. If not, I’ll be syncing the HDFS, YARN, & MR pre commit scripts. (right now, there are slight differences between them. Hooray!) Thanks.
