Answering myself: Flaky Test Report
> On 2020. Aug 14., at 12:52, Andor Molnar <an...@apache.org> wrote: > > Hi devs, > > There’s some movement from Apache on Jenkins migration. Github account is now > available on the new instance which will be useful for PreCommit builds. I > started the tests. > > Deadline (15 Aug) is approaching. Is there anything else which has fallen > through the cracks and has to be migrated to the new instance? > > Andor > > > >> On 2020. Jul 24., at 19:15, Patrick Hunt <ph...@apache.org> wrote: >> >> On Fri, Jul 24, 2020 at 3:05 AM Andor Molnar <an...@apache.org> wrote: >> >>> Hi, >>> >>> Jenkinsfile is now committed to master, 3.5 and 3.6 branches. Build is >>> running here: >>> >>> https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-master-maven-multipipeline/ >>> >>> I use standard Git interface to access the repo, because using GitHub >>> means that branch discovery has to run through the github API. The problem >>> with this is that I haven’t specified credentials and anonymous access is >>> limited by some quota, meaning the discovery process takes ages (1-2 hours). >>> >>> Builds are running fine except I see a lot of concurrent builds on master >>> without any meaningful SCM change. Odd. I will change pollSCM trigger to >>> run @hourly and will see how it goes. >>> >>> What’s outstanding? >>> >>> - JDKs: currently only two: 8, 11 (LTS versions). What other JDKs would >>> you like to run against? >>> >>> - GitHub Pull Requests precommit job: I have no idea how to do this, but I >>> suspect we have to use the GitHub Api for this to work. >>> >>> >> Not sure if this is helpful but I saw it recently: >> >>> we need ability to trigger builds on pull request creation/updates which >> >>> requires the plug-in below or similar: >> >>> https://plugins.jenkins.io/ghprb/ >> >> >> >> That plugin is deprecated and not recommended for use . >> >> >> I have installed the *branch source plugin instead*, hopefully that works >> for >> >> you. >> >> >> If not, let's investigate further what else we can do >> >> >> >>> - Windows build: there’s no Windows agent available currently in the new >>> instance. >>> >>> - Ant builds: do we need this? >>> >>> Please share your thoughts. >>> >>> Regards, >>> Andor >>> >>> >>> >>> >>>> On 2020. Jul 23., at 22:08, Andor Molnar <an...@apache.org> wrote: >>>> >>>> Hi folks, >>>> >>>> PR is available: https://github.com/apache/zookeeper/pull/1409 >>>> >>>> Andor >>>> >>>> >>>> >>>> On Thu, 2020-07-23 at 11:32 +0200, Andor Molnar wrote: >>>>> Created a Jira for the task: >>>>> >>>>> https://issues.apache.org/jira/browse/ZOOKEEPER-3896 >>>>> >>>>> Andor >>>>> >>>>> >>>>> >>>>>> On 2020. Jul 21., at 12:10, Andor Molnar <an...@apache.org> wrote: >>>>>> >>>>>> Where’s that example…? >>>>>> >>>>>> >>>>>> >>>>>>> On 2020. Jul 20., at 20:02, Enrico Olivelli <eolive...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>> Il Lun 20 Lug 2020, 19:40 Andor Molnar <an...@apache.org> ha >>>>>>> scritto: >>>>>>> >>>>>>>> Hi Enrico, >>>>>>>> >>>>>>>> No worries, I only created a few jobs to make some progress, >>>>>>>> but feel free >>>>>>>> to ignore that and do it in a better way. The “View” or >>>>>>>> “Folder” that I was >>>>>>>> adding jobs is >>>>>>>> >>>>>>>> https://ci-hadoop.apache.org/view/ZooKeeper/ >>>>>>>> >>>>>>>> >>>>>>>> Andor >>>>>>>> >>>>>>> >>>>>>> This is an example from Apache Maven project. It is very complex >>>>>>> because >>>>>>> tests are in another repo and for lots if other reasons. We just >>>>>>> have to >>>>>>> create a simpler file. >>>>>>> >>>>>>> If nobody volunteers I can try to spend some time but I won't >>>>>>> have a fast >>>>>>> pace these weeks >>>>>>> >>>>>>> >>>>>>> Enrico >>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>>>> On 2020. Jul 20., at 19:34, Enrico Olivelli < >>>>>>>>> eolive...@gmail.com> wrote: >>>>>>>>> >>>>>>>>> Il Lun 20 Lug 2020, 19:02 Patrick Hunt <ph...@apache.org> ha >>>>>>>>> scritto: >>>>>>>>> >>>>>>>>>> On Mon, Jul 20, 2020 at 9:47 AM Enrico Olivelli < >>>>>>>>>> eolive...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Il Lun 20 Lug 2020, 18:41 Patrick Hunt <ph...@apache.org> >>>>>>>>>>> ha scritto: >>>>>>>>>>> >>>>>>>>>>>> On Sat, Jul 18, 2020 at 12:20 PM Andor Molnar < >>>>>>>>>>>> an...@apache.org> >>>>>>>>>> wrote: >>>>>>>>>>>>> Hi Pat, >>>>>>>>>>>>> >>>>>>>>>>>>> I have admin rights in the new system too and >>>>>>>>>>>>> probably can work on >>>>>>>>>> this >>>>>>>>>>>> on >>>>>>>>>>>>> Monday. >>>>>>>>>>>>> What’s “matrix” config? Shouldn’t we just replicate >>>>>>>>>>>>> the same jobs on >>>>>>>>>>> the >>>>>>>>>>>>> new instance? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> Not sure on the exact name/feature but "matrix" is >>>>>>>>>>>> basically the >>>>>>>>>> ability >>>>>>>>>>> to >>>>>>>>>>>> say "run this build/test across a set of JDK versions" >>>>>>>>>>>> rather than a >>>>>>>>>>> single >>>>>>>>>>>> version. As a result, instead of 3 jobs for zk3.6.0, >>>>>>>>>>>> jdk 1/2/3 you end >>>>>>>>>> up >>>>>>>>>>>> with a single job which runs three times, one for each >>>>>>>>>>>> jdk type and >>>>>>>>>>>> summarizes the results. I've seen this before, I assume >>>>>>>>>>>> it's a feature >>>>>>>>>> of >>>>>>>>>>>> jenkins itself? >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> We should use Jenkins files and this configuration will >>>>>>>>>>> be easy and >>>>>>>>>>> committed to git >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The ability to do "gitops" would be amazing! >>>>>>>>>> >>>>>>>>> >>>>>>>>> Yes. >>>>>>>>> But I saw on Slack that Andor already started to create new >>>>>>>>> jobs. >>>>>>>>> We should do only one way in order not to waste time. >>>>>>>>> >>>>>>>>> Andor can you please share your work? >>>>>>>>> Thanks for doing it >>>>>>>>> >>>>>>>>> Enrico >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> Patrick >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Enrico >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Patrick >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> Andor >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> On 2020. Jul 17., at 2:51, Patrick Hunt < >>>>>>>>>>>>>> ph...@apache.org> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> I updated the job I linked earlier based on what's >>>>>>>>>>>>>> the latest on >>>>>>>>>> the >>>>>>>>>>>>> legacy >>>>>>>>>>>>>> jenkins. It ran successfully >>>>>>>>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> I didn't replicate every config setting - main gap >>>>>>>>>>>>>> is the spotbugs >>>>>>>>>>>>>> post-build, which seems to be missing from the new >>>>>>>>>>>>>> jenkins plugins. >>>>>>>>>>>>>> >>>>>>>>>>>>>> That's just maven master though. Not sure about the >>>>>>>>>>>>>> rest. Can we do >>>>>>>>>>>> more >>>>>>>>>>>>> of >>>>>>>>>>>>>> a "matrix" config in the new system vs cloning all >>>>>>>>>>>>>> the time? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Patrick >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt < >>>>>>>>>>>>>> ph...@apache.org> >>>>>>>>>>> wrote: >>>>>>>>>>>>>>> On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt < >>>>>>>>>>>>>>> ph...@apache.org> >>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Thu, Jul 16, 2020 at 12:56 PM Enrico >>>>>>>>>>>>>>>> Olivelli < >>>>>>>>>>>> eolive...@gmail.com> >>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> FYI >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Fun. I do notice it says "Hadoop and related >>>>>>>>>>>>>>>> projects have their >>>>>>>>>>> own >>>>>>>>>>>>>>>> migration path to follow" - any insight on >>>>>>>>>>>>>>>> that? We are or are >>>>>>>>>> not >>>>>>>>>>>>> lumped >>>>>>>>>>>>>>>> in? I would assume we are? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> This (eventual migration) came up a while back >>>>>>>>>>>>>>>> on the Hadoop PMC >>>>>>>>>>> and >>>>>>>>>>>> I >>>>>>>>>>>>>>>> volunteered to try for us (ZK). I was never >>>>>>>>>>>>>>>> able to get it to >>>>>>>>>>> work, I >>>>>>>>>>>>>>>> provided feedback to infra but they never got >>>>>>>>>>>>>>>> back, as such we >>>>>>>>>>> have >>>>>>>>>>>> a >>>>>>>>>>>>>>>> project here that's not working with some basic >>>>>>>>>>>>>>>> dependencies >>>>>>>>>>> missing: >>>>>>>>>>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> That said, we can try again. Can we verify >>>>>>>>>>>>>>>> where ZK is supposed >>>>>>>>>> to >>>>>>>>>>>>> land? >>>>>>>>>>>>>>>> Perhaps we can try to delete and recreate the >>>>>>>>>>>>>>>> POC job I created >>>>>>>>>> at >>>>>>>>>>>> that >>>>>>>>>>>>>>>> link to see if we can get it working? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I see another email thread on the list saying >>>>>>>>>>>>>>> that we are part of >>>>>>>>>>> said >>>>>>>>>>>>>>> "related projects". We are expected to move to >>>>>>>>>>>>>>> http://ci-hadoop.apache.org/ within 4 weeks. >>>>>>>>>>>>>>> Seems nodes are >>>>>>>>>>> already >>>>>>>>>>>>>>> being removed/migrated from the "H#" pool. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Also this: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> There are over 400 plugins on the current >>>>>>>>>>>>>>> builds.apache.org - >>>>>>>>>> most >>>>>>>>>>> of >>>>>>>>>>>>> which >>>>>>>>>>>>>>> we don't need any more, or are replaced with >>>>>>>>>>>>>>> different plugins on >>>>>>>>>>> the >>>>>>>>>>>>> new >>>>>>>>>>>>>>> system. I expect there may be some plugins we >>>>>>>>>>>>>>> still need to >>>>>>>>>> install >>>>>>>>>>> to >>>>>>>>>>>>> get >>>>>>>>>>>>>>> you going again, which is why it is vitally >>>>>>>>>>>>>>> important that you >>>>>>>>>> start >>>>>>>>>>>>>>> testing and migrating your jobs over *now*. You >>>>>>>>>>>>>>> should all have >>>>>>>>>>> auth. >>>>>>>>>>>>>>> Any questions, feel free to email the >>>>>>>>>> hadoop-migrati...@apache.org >>>>>>>>>>>>> list if >>>>>>>>>>>>>>> you are one of the projects listed below. The >>>>>>>>>>>>>>> rest of you, not >>>>>>>>>>>> listed, a >>>>>>>>>>>>>>> similar email to this one will be posted for you >>>>>>>>>>>>>>> shortly on >>>>>>>>>>>> builds@a.o. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> full details >>>>>>>>>>>>>>> >>>>>>>> >>> https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E >>>>>>>>>>>>>>> Patrick >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Patrick >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ---------- Forwarded message --------- >>>>>>>>>>>>>>>>> Da: Gavin McDonald <gmcdon...@apache.org> >>>>>>>>>>>>>>>>> Date: Gio 16 Lug 2020, 18:33 >>>>>>>>>>>>>>>>> Subject: [IMPORTANT] - Migration to ci- >>>>>>>>>>>>>>>>> builds.a.o >>>>>>>>>>>>>>>>> To: builds <bui...@apache.org> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> This NOTICE is for everyone on >>>>>>>>>>>>>>>>> builds.apache.org. We are >>>>>>>>>>> migrating >>>>>>>>>>>>> to a >>>>>>>>>>>>>>>>> new >>>>>>>>>>>>>>>>> Cloudbees based Client Master called >>>>>>>>>> https://ci-builds.apache.org >>>>>>>>>>> . >>>>>>>>>>>>> The >>>>>>>>>>>>>>>>> migrations of all jobs needs to be done >>>>>>>>>>>>>>>>> before the switch off >>>>>>>>>> date >>>>>>>>>>>> of >>>>>>>>>>>>>>>>> 15th >>>>>>>>>>>>>>>>> August 2020, so you have a maximum of 4 >>>>>>>>>>>>>>>>> weeks. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> There is no tool or automated way of >>>>>>>>>>>>>>>>> migrating your jobs, the >>>>>>>>>>>>>>>>> differences in the platforms, the plugins and >>>>>>>>>>>>>>>>> the setup makes it >>>>>>>>>>>>>>>>> impossible >>>>>>>>>>>>>>>>> to do in a safe way. So, you all need to >>>>>>>>>>>>>>>>> start creating new jobs >>>>>>>>>>> on >>>>>>>>>>>>>>>>> ci-infra.a.o and then , when you are happy, >>>>>>>>>>>>>>>>> turn off your old >>>>>>>>>>> builds >>>>>>>>>>>>> on >>>>>>>>>>>>>>>>> builds.a.o. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> There are currently 4 agents over there ready >>>>>>>>>>>>>>>>> to take jobs, >>>>>>>>>> plus a >>>>>>>>>>>>>>>>> floating >>>>>>>>>>>>>>>>> agent which is shared amongst many masters >>>>>>>>>>>>>>>>> (more to come). I >>>>>>>>>> will >>>>>>>>>>>>> migrate >>>>>>>>>>>>>>>>> away 2 more agents from builds.a.o to ci- >>>>>>>>>>>>>>>>> builds.a.o every few >>>>>>>>>>> days, >>>>>>>>>>>>> and >>>>>>>>>>>>>>>>> will keep an eye of load across both and >>>>>>>>>>>>>>>>> adjust accordingly. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> If needed, create a ticket on INFRA jira for >>>>>>>>>>>>>>>>> any issues that >>>>>>>>>> crop >>>>>>>>>>>> up, >>>>>>>>>>>>> or >>>>>>>>>>>>>>>>> email here on builds@a.o. there may be one or >>>>>>>>>>>>>>>>> two plugins we >>>>>>>>>> need >>>>>>>>>>>> to >>>>>>>>>>>>>>>>> install/tweak etc. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> We will be not using 'Views' at the top >>>>>>>>>>>>>>>>> level, but rather we >>>>>>>>>> will >>>>>>>>>>>> take >>>>>>>>>>>>>>>>> advantage of 'Folders Plus' - each project >>>>>>>>>>>>>>>>> will get its own >>>>>>>>>> Folder >>>>>>>>>>>> and >>>>>>>>>>>>>>>>> have >>>>>>>>>>>>>>>>> authorisation access to create/edit jobs etc >>>>>>>>>>>>>>>>> within that >>>>>>>>>> folder. I >>>>>>>>>>>>> will >>>>>>>>>>>>>>>>> create these folders as you ask for them to >>>>>>>>>>>>>>>>> start with. This >>>>>>>>>> setup >>>>>>>>>>>>> allows >>>>>>>>>>>>>>>>> for credentials isolation amongst other >>>>>>>>>>>>>>>>> benefits, including but >>>>>>>>>>> not >>>>>>>>>>>>>>>>> limited >>>>>>>>>>>>>>>>> to exclusive agents (Controlled Agents) for >>>>>>>>>>>>>>>>> your own use , >>>>>>>>>> should >>>>>>>>>>>> you >>>>>>>>>>>>>>>>> have >>>>>>>>>>>>>>>>> any project targeted donations of agents. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> As with other aspects of the ASF, projects >>>>>>>>>>>>>>>>> can choose to just >>>>>>>>>>> enable >>>>>>>>>>>>> all >>>>>>>>>>>>>>>>> committers access to their folder, just ask. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> We will re-use builds.apache.org as a CNAME >>>>>>>>>>>>>>>>> to ci-builds.a.o >>>>>>>>>> but >>>>>>>>>>>> will >>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>> be setting up any forwarding rules or >>>>>>>>>>>>>>>>> anything like that. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> So, please, get started *now *on this so you >>>>>>>>>>>>>>>>> can be sure we are >>>>>>>>>>> all >>>>>>>>>>>>>>>>> completed before the final cutoff date of >>>>>>>>>>>>>>>>> 15th August 2020. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Any questions - I expect a few (dozen :) ) - >>>>>>>>>>>>>>>>> ask away and/or >>>>>>>>>> file >>>>>>>>>>>>> INFRA >>>>>>>>>>>>>>>>> tickets. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hadoop and related projects have their own >>>>>>>>>>>>>>>>> migration path to >>>>>>>>>>> follow, >>>>>>>>>>>>> same >>>>>>>>>>>>>>>>> cut off date, Cassandra, Beam, CouchDB have >>>>>>>>>>>>>>>>> already migrated and >>>>>>>>>>> are >>>>>>>>>>>>>>>>> doing >>>>>>>>>>>>>>>>> very well in their new homes. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Lets get going ... >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> *Gavin McDonald* >>>>>>>>>>>>>>>>> Systems Administrator >>>>>>>>>>>>>>>>> ASF Infrastructure Team >>>> >>> >>> >
