Hi, Jenkinsfile is now committed to master, 3.5 and 3.6 branches. Build is running here: https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-master-maven-multipipeline/
I use standard Git interface to access the repo, because using GitHub means that branch discovery has to run through the github API. The problem with this is that I haven’t specified credentials and anonymous access is limited by some quota, meaning the discovery process takes ages (1-2 hours). Builds are running fine except I see a lot of concurrent builds on master without any meaningful SCM change. Odd. I will change pollSCM trigger to run @hourly and will see how it goes. What’s outstanding? - JDKs: currently only two: 8, 11 (LTS versions). What other JDKs would you like to run against? - GitHub Pull Requests precommit job: I have no idea how to do this, but I suspect we have to use the GitHub Api for this to work. - Windows build: there’s no Windows agent available currently in the new instance. - Ant builds: do we need this? Please share your thoughts. Regards, Andor > On 2020. Jul 23., at 22:08, Andor Molnar <an...@apache.org> wrote: > > Hi folks, > > PR is available: https://github.com/apache/zookeeper/pull/1409 > > Andor > > > > On Thu, 2020-07-23 at 11:32 +0200, Andor Molnar wrote: >> Created a Jira for the task: >> >> https://issues.apache.org/jira/browse/ZOOKEEPER-3896 >> >> Andor >> >> >> >>> On 2020. Jul 21., at 12:10, Andor Molnar <an...@apache.org> wrote: >>> >>> Where’s that example…? >>> >>> >>> >>>> On 2020. Jul 20., at 20:02, Enrico Olivelli <eolive...@gmail.com> >>>> wrote: >>>> >>>> Il Lun 20 Lug 2020, 19:40 Andor Molnar <an...@apache.org> ha >>>> scritto: >>>> >>>>> Hi Enrico, >>>>> >>>>> No worries, I only created a few jobs to make some progress, >>>>> but feel free >>>>> to ignore that and do it in a better way. The “View” or >>>>> “Folder” that I was >>>>> adding jobs is >>>>> >>>>> https://ci-hadoop.apache.org/view/ZooKeeper/ >>>>> >>>>> >>>>> Andor >>>>> >>>> >>>> This is an example from Apache Maven project. It is very complex >>>> because >>>> tests are in another repo and for lots if other reasons. We just >>>> have to >>>> create a simpler file. >>>> >>>> If nobody volunteers I can try to spend some time but I won't >>>> have a fast >>>> pace these weeks >>>> >>>> >>>> Enrico >>>> >>>> >>>> >>>>> >>>>>> On 2020. Jul 20., at 19:34, Enrico Olivelli < >>>>>> eolive...@gmail.com> wrote: >>>>>> >>>>>> Il Lun 20 Lug 2020, 19:02 Patrick Hunt <ph...@apache.org> ha >>>>>> scritto: >>>>>> >>>>>>> On Mon, Jul 20, 2020 at 9:47 AM Enrico Olivelli < >>>>>>> eolive...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Il Lun 20 Lug 2020, 18:41 Patrick Hunt <ph...@apache.org> >>>>>>>> ha scritto: >>>>>>>> >>>>>>>>> On Sat, Jul 18, 2020 at 12:20 PM Andor Molnar < >>>>>>>>> an...@apache.org> >>>>>>> wrote: >>>>>>>>>> Hi Pat, >>>>>>>>>> >>>>>>>>>> I have admin rights in the new system too and >>>>>>>>>> probably can work on >>>>>>> this >>>>>>>>> on >>>>>>>>>> Monday. >>>>>>>>>> What’s “matrix” config? Shouldn’t we just replicate >>>>>>>>>> the same jobs on >>>>>>>> the >>>>>>>>>> new instance? >>>>>>>>>> >>>>>>>>>> >>>>>>>>> Not sure on the exact name/feature but "matrix" is >>>>>>>>> basically the >>>>>>> ability >>>>>>>> to >>>>>>>>> say "run this build/test across a set of JDK versions" >>>>>>>>> rather than a >>>>>>>> single >>>>>>>>> version. As a result, instead of 3 jobs for zk3.6.0, >>>>>>>>> jdk 1/2/3 you end >>>>>>> up >>>>>>>>> with a single job which runs three times, one for each >>>>>>>>> jdk type and >>>>>>>>> summarizes the results. I've seen this before, I assume >>>>>>>>> it's a feature >>>>>>> of >>>>>>>>> jenkins itself? >>>>>>>>> >>>>>>>> >>>>>>>> We should use Jenkins files and this configuration will >>>>>>>> be easy and >>>>>>>> committed to git >>>>>>>> >>>>>>> >>>>>>> The ability to do "gitops" would be amazing! >>>>>>> >>>>>> >>>>>> Yes. >>>>>> But I saw on Slack that Andor already started to create new >>>>>> jobs. >>>>>> We should do only one way in order not to waste time. >>>>>> >>>>>> Andor can you please share your work? >>>>>> Thanks for doing it >>>>>> >>>>>> Enrico >>>>>> >>>>>> >>>>>> >>>>>>> Patrick >>>>>>> >>>>>>> >>>>>>>> Enrico >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> Patrick >>>>>>>>> >>>>>>>>> >>>>>>>>>> Andor >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On 2020. Jul 17., at 2:51, Patrick Hunt < >>>>>>>>>>> ph...@apache.org> wrote: >>>>>>>>>>> >>>>>>>>>>> I updated the job I linked earlier based on what's >>>>>>>>>>> the latest on >>>>>>> the >>>>>>>>>> legacy >>>>>>>>>>> jenkins. It ran successfully >>>>>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/ >>>>>>>>>>> >>>>>>>>>>> I didn't replicate every config setting - main gap >>>>>>>>>>> is the spotbugs >>>>>>>>>>> post-build, which seems to be missing from the new >>>>>>>>>>> jenkins plugins. >>>>>>>>>>> >>>>>>>>>>> That's just maven master though. Not sure about the >>>>>>>>>>> rest. Can we do >>>>>>>>> more >>>>>>>>>> of >>>>>>>>>>> a "matrix" config in the new system vs cloning all >>>>>>>>>>> the time? >>>>>>>>>>> >>>>>>>>>>> Patrick >>>>>>>>>>> >>>>>>>>>>> On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt < >>>>>>>>>>> ph...@apache.org> >>>>>>>> wrote: >>>>>>>>>>>> On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt < >>>>>>>>>>>> ph...@apache.org> >>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> On Thu, Jul 16, 2020 at 12:56 PM Enrico >>>>>>>>>>>>> Olivelli < >>>>>>>>> eolive...@gmail.com> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> FYI >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Fun. I do notice it says "Hadoop and related >>>>>>>>>>>>> projects have their >>>>>>>> own >>>>>>>>>>>>> migration path to follow" - any insight on >>>>>>>>>>>>> that? We are or are >>>>>>> not >>>>>>>>>> lumped >>>>>>>>>>>>> in? I would assume we are? >>>>>>>>>>>>> >>>>>>>>>>>>> This (eventual migration) came up a while back >>>>>>>>>>>>> on the Hadoop PMC >>>>>>>> and >>>>>>>>> I >>>>>>>>>>>>> volunteered to try for us (ZK). I was never >>>>>>>>>>>>> able to get it to >>>>>>>> work, I >>>>>>>>>>>>> provided feedback to infra but they never got >>>>>>>>>>>>> back, as such we >>>>>>>> have >>>>>>>>> a >>>>>>>>>>>>> project here that's not working with some basic >>>>>>>>>>>>> dependencies >>>>>>>> missing: >>>>>>>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/ >>>>>>>>>>>>> >>>>>>>>>>>>> That said, we can try again. Can we verify >>>>>>>>>>>>> where ZK is supposed >>>>>>> to >>>>>>>>>> land? >>>>>>>>>>>>> Perhaps we can try to delete and recreate the >>>>>>>>>>>>> POC job I created >>>>>>> at >>>>>>>>> that >>>>>>>>>>>>> link to see if we can get it working? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> I see another email thread on the list saying >>>>>>>>>>>> that we are part of >>>>>>>> said >>>>>>>>>>>> "related projects". We are expected to move to >>>>>>>>>>>> http://ci-hadoop.apache.org/ within 4 weeks. >>>>>>>>>>>> Seems nodes are >>>>>>>> already >>>>>>>>>>>> being removed/migrated from the "H#" pool. >>>>>>>>>>>> >>>>>>>>>>>> Also this: >>>>>>>>>>>> >>>>>>>>>>>> There are over 400 plugins on the current >>>>>>>>>>>> builds.apache.org - >>>>>>> most >>>>>>>> of >>>>>>>>>> which >>>>>>>>>>>> we don't need any more, or are replaced with >>>>>>>>>>>> different plugins on >>>>>>>> the >>>>>>>>>> new >>>>>>>>>>>> system. I expect there may be some plugins we >>>>>>>>>>>> still need to >>>>>>> install >>>>>>>> to >>>>>>>>>> get >>>>>>>>>>>> you going again, which is why it is vitally >>>>>>>>>>>> important that you >>>>>>> start >>>>>>>>>>>> testing and migrating your jobs over *now*. You >>>>>>>>>>>> should all have >>>>>>>> auth. >>>>>>>>>>>> Any questions, feel free to email the >>>>>>> hadoop-migrati...@apache.org >>>>>>>>>> list if >>>>>>>>>>>> you are one of the projects listed below. The >>>>>>>>>>>> rest of you, not >>>>>>>>> listed, a >>>>>>>>>>>> similar email to this one will be posted for you >>>>>>>>>>>> shortly on >>>>>>>>> builds@a.o. >>>>>>>>>>>> >>>>>>>>>>>> full details >>>>>>>>>>>> >>>>> https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E >>>>>>>>>>>> Patrick >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> Patrick >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> ---------- Forwarded message --------- >>>>>>>>>>>>>> Da: Gavin McDonald <gmcdon...@apache.org> >>>>>>>>>>>>>> Date: Gio 16 Lug 2020, 18:33 >>>>>>>>>>>>>> Subject: [IMPORTANT] - Migration to ci- >>>>>>>>>>>>>> builds.a.o >>>>>>>>>>>>>> To: builds <bui...@apache.org> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>> >>>>>>>>>>>>>> This NOTICE is for everyone on >>>>>>>>>>>>>> builds.apache.org. We are >>>>>>>> migrating >>>>>>>>>> to a >>>>>>>>>>>>>> new >>>>>>>>>>>>>> Cloudbees based Client Master called >>>>>>> https://ci-builds.apache.org >>>>>>>> . >>>>>>>>>> The >>>>>>>>>>>>>> migrations of all jobs needs to be done >>>>>>>>>>>>>> before the switch off >>>>>>> date >>>>>>>>> of >>>>>>>>>>>>>> 15th >>>>>>>>>>>>>> August 2020, so you have a maximum of 4 >>>>>>>>>>>>>> weeks. >>>>>>>>>>>>>> >>>>>>>>>>>>>> There is no tool or automated way of >>>>>>>>>>>>>> migrating your jobs, the >>>>>>>>>>>>>> differences in the platforms, the plugins and >>>>>>>>>>>>>> the setup makes it >>>>>>>>>>>>>> impossible >>>>>>>>>>>>>> to do in a safe way. So, you all need to >>>>>>>>>>>>>> start creating new jobs >>>>>>>> on >>>>>>>>>>>>>> ci-infra.a.o and then , when you are happy, >>>>>>>>>>>>>> turn off your old >>>>>>>> builds >>>>>>>>>> on >>>>>>>>>>>>>> builds.a.o. >>>>>>>>>>>>>> >>>>>>>>>>>>>> There are currently 4 agents over there ready >>>>>>>>>>>>>> to take jobs, >>>>>>> plus a >>>>>>>>>>>>>> floating >>>>>>>>>>>>>> agent which is shared amongst many masters >>>>>>>>>>>>>> (more to come). I >>>>>>> will >>>>>>>>>> migrate >>>>>>>>>>>>>> away 2 more agents from builds.a.o to ci- >>>>>>>>>>>>>> builds.a.o every few >>>>>>>> days, >>>>>>>>>> and >>>>>>>>>>>>>> will keep an eye of load across both and >>>>>>>>>>>>>> adjust accordingly. >>>>>>>>>>>>>> >>>>>>>>>>>>>> If needed, create a ticket on INFRA jira for >>>>>>>>>>>>>> any issues that >>>>>>> crop >>>>>>>>> up, >>>>>>>>>> or >>>>>>>>>>>>>> email here on builds@a.o. there may be one or >>>>>>>>>>>>>> two plugins we >>>>>>> need >>>>>>>>> to >>>>>>>>>>>>>> install/tweak etc. >>>>>>>>>>>>>> >>>>>>>>>>>>>> We will be not using 'Views' at the top >>>>>>>>>>>>>> level, but rather we >>>>>>> will >>>>>>>>> take >>>>>>>>>>>>>> advantage of 'Folders Plus' - each project >>>>>>>>>>>>>> will get its own >>>>>>> Folder >>>>>>>>> and >>>>>>>>>>>>>> have >>>>>>>>>>>>>> authorisation access to create/edit jobs etc >>>>>>>>>>>>>> within that >>>>>>> folder. I >>>>>>>>>> will >>>>>>>>>>>>>> create these folders as you ask for them to >>>>>>>>>>>>>> start with. This >>>>>>> setup >>>>>>>>>> allows >>>>>>>>>>>>>> for credentials isolation amongst other >>>>>>>>>>>>>> benefits, including but >>>>>>>> not >>>>>>>>>>>>>> limited >>>>>>>>>>>>>> to exclusive agents (Controlled Agents) for >>>>>>>>>>>>>> your own use , >>>>>>> should >>>>>>>>> you >>>>>>>>>>>>>> have >>>>>>>>>>>>>> any project targeted donations of agents. >>>>>>>>>>>>>> >>>>>>>>>>>>>> As with other aspects of the ASF, projects >>>>>>>>>>>>>> can choose to just >>>>>>>> enable >>>>>>>>>> all >>>>>>>>>>>>>> committers access to their folder, just ask. >>>>>>>>>>>>>> >>>>>>>>>>>>>> We will re-use builds.apache.org as a CNAME >>>>>>>>>>>>>> to ci-builds.a.o >>>>>>> but >>>>>>>>> will >>>>>>>>>>>>>> not >>>>>>>>>>>>>> be setting up any forwarding rules or >>>>>>>>>>>>>> anything like that. >>>>>>>>>>>>>> >>>>>>>>>>>>>> So, please, get started *now *on this so you >>>>>>>>>>>>>> can be sure we are >>>>>>>> all >>>>>>>>>>>>>> completed before the final cutoff date of >>>>>>>>>>>>>> 15th August 2020. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Any questions - I expect a few (dozen :) ) - >>>>>>>>>>>>>> ask away and/or >>>>>>> file >>>>>>>>>> INFRA >>>>>>>>>>>>>> tickets. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hadoop and related projects have their own >>>>>>>>>>>>>> migration path to >>>>>>>> follow, >>>>>>>>>> same >>>>>>>>>>>>>> cut off date, Cassandra, Beam, CouchDB have >>>>>>>>>>>>>> already migrated and >>>>>>>> are >>>>>>>>>>>>>> doing >>>>>>>>>>>>>> very well in their new homes. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Lets get going ... >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> >>>>>>>>>>>>>> *Gavin McDonald* >>>>>>>>>>>>>> Systems Administrator >>>>>>>>>>>>>> ASF Infrastructure Team >
