Thank you Gavin. WIll take a look soon! Gavin McDonald <[email protected]> 于2022年2月8日周二 18:31写道：
> Give it a try now please > > On Sat, Feb 5, 2022 at 8:55 AM 张铎(Duo Zhang) <[email protected]> > wrote: > > > Hi, Gavin, any luck on the ssh transfer plugin? > > > > Thanks. > > > > 张铎(Duo Zhang) <[email protected]> 于2022年1月29日周六 14:53写道： > > > >> So any updates here Gavin? > >> > >> Is the ssh transfer plugin available now? > >> > >> Thanks. > >> > >> Sean Busbey <[email protected]> 于2022年1月23日周日 09:04写道： > >> > >>> It looks like I currently do have access to manage the ci-hbase > >>> controller. > >>> > >>> > >>> Could you check that the git-websites label handling where a common > >>> machine gets leased to the ci-hbase controller is set up properly? or > give > >>> me a pointer on how I would check this myself? > >>> > >>> I started trying to migrate our website generation job again and it's > >>> been waiting for a node with the git-websites label to show up for ~8 > hours > >>> at this point. > >>> > >>> https://ci-hbase.apache.org/job/hbase_generate_website/3/ > >>> > >>> > >>> > >>> On Sat, Jan 22, 2022 at 2:42 AM Gavin McDonald <[email protected]> > >>> wrote: > >>> > >>>> Hi, > >>>> Timestamper plugin installed. > >>>> > >>>> By the way I thought I had given the PMC members permissions to manage > >>>> Jenkins > >>>> on ci-hbase including installing plugins etc, can someone check? > >>>> > >>>> > >>>> On Sat, Jan 22, 2022 at 7:34 AM 张铎(Duo Zhang) <[email protected]> > >>>> wrote: > >>>> > >>>> > We faced a strange issue.. > >>>> > > >>>> > rg.codehaus.groovy.control.MultipleCompilationErrorsException: > >>>> startup failed: > >>>> > WorkflowScript: 29: Invalid option type "timestamps". Valid option > >>>> types: [buildDiscarder, catchError, checkoutToSubdirectory, > >>>> disableConcurrentBuilds, disableResume, durabilityHint, > >>>> githubProjectProperty, newContainerPerStage, overrideIndexTriggers, > >>>> parallelsAlwaysFailFast, preserveStashes, quietPeriod, > rateLimitBuilds, > >>>> retry, script, skipDefaultCheckout, skipStagesAfterUnstable, timeout, > >>>> waitUntil, warnError, withChecks, withContext, withCredentials, > withEnv, > >>>> wrap, ws] @ line 29, column 5. > >>>> > timestamps() > >>>> > ^ > >>>> > > >>>> > 1 error > >>>> > > >>>> > at > >>>> > org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310) > >>>> > at > >>>> > org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085) > >>>> > at > >>>> > org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603) > >>>> > at > >>>> > org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581) > >>>> > at > >>>> > org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558) > >>>> > at > >>>> groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298) > >>>> > at > >>>> groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268) > >>>> > at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688) > >>>> > at groovy.lang.GroovyShell.parse(GroovyShell.java:700) > >>>> > at > >>>> > org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142) > >>>> > at > >>>> > org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127) > >>>> > at > >>>> > org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:571) > >>>> > at > >>>> > org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:523) > >>>> > at > >>>> > org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:334) > >>>> > at > >>>> hudson.model.ResourceController.execute(ResourceController.java:97) > >>>> > at hudson.model.Executor.run(Executor.java:429) > >>>> > > >>>> > > >>>> > After googling, it seems a restart of jenkins can fix it, or we do > not > >>>> > have timestamper plugin installed? Mind taking a look Gavin? > >>>> > > >>>> > Thanks. > >>>> > > >>>> > 张铎(Duo Zhang) <[email protected]> 于2022年1月22日周六 11:40写道： > >>>> > > >>>> >> Oh, just saw that we could do this by ourselves. > >>>> >> > >>>> >> Let me have a try. > >>>> >> > >>>> >> 张铎(Duo Zhang) <[email protected]> 于2022年1月22日周六 11:18写道： > >>>> >> > >>>> >>> OK, so while copying you could keep the old jenkins builds? > >>>> >>> That's awesome. > >>>> >>> > >>>> >>> Then please help 'copying' HBase-Find-Flaky-Tests and > >>>> HBase-Flaky-Tests > >>>> >>> to ci-hbase too? We need the history builds to calucate the flaky > >>>> test list > >>>> >>> so if all the old build data can be kept then it will help us a > lot. > >>>> >>> > >>>> >>> Thanks~ > >>>> >>> > >>>> >>> 张铎(Duo Zhang) <[email protected]> 于2022年1月22日周六 11:03写道： > >>>> >>> > >>>> >>>> Thank you Gavin. > >>>> >>>> > >>>> >>>> Let me first move some light weighted job first to see if > >>>> everything is > >>>> >>>> OK. > >>>> >>>> > >>>> >>>> HBase Nightly, some flaky related jobs, and also the main pre > >>>> commit > >>>> >>>> job for HBase are tied together. We need to move them at once. > >>>> >>>> > >>>> >>>> Gavin McDonald <[email protected]> 于2022年1月22日周六 02:44写道： > >>>> >>>> > >>>> >>>>> Thanks for your patience > >>>> >>>>> > >>>> >>>>> I have 'copied' over a job - > >>>> >>>>> https://ci-hbase.apache.org/job/HBase%20Nightly/ > >>>> >>>>> > >>>> >>>>> which of course made me aware of what plugins I needed to > install. > >>>> >>>>> > >>>> >>>>> Feel free to delete that, or use it and delete the old, and let > me > >>>> >>>>> know if have any issues > >>>> >>>>> with any other missing plugins. > >>>> >>>>> > >>>> >>>>> > >>>> >>>>> > >>>> >>>>> On Thu, Jan 20, 2022 at 1:11 PM 张铎(Duo Zhang) < > >>>> [email protected]> > >>>> >>>>> wrote: > >>>> >>>>> > >>>> >>>>>> I’ve also replied on the jira issue. > >>>> >>>>>> > >>>> >>>>>> You can try to copy a job to the new Jenkins. > >>>> >>>>>> > >>>> >>>>>> Thanks. > >>>> >>>>>> > >>>> >>>>>> Gavin McDonald <[email protected]>于2022年1月20日 周四18:45写道： > >>>> >>>>>> > >>>> >>>>>>> Im taking a look ... > >>>> >>>>>>> > >>>> >>>>>>> On Thu, Jan 20, 2022 at 6:54 AM Sean Busbey < > [email protected]> > >>>> >>>>>>> wrote: > >>>> >>>>>>> > >>>> >>>>>>> > Hi Gavin > >>>> >>>>>>> > > >>>> >>>>>>> > I went to start migrating jobs to the ci-hbase controller, > >>>> but the > >>>> >>>>>>> new job > >>>> >>>>>>> > workflow doesn't include pipelines or multibranch pipelines. > >>>> We > >>>> >>>>>>> use both of > >>>> >>>>>>> > these job types for nearly all of our work. Can they be > >>>> enabled? > >>>> >>>>>>> > > >>>> >>>>>>> > On Tue, Jan 18, 2022 at 5:04 AM Gavin McDonald < > >>>> >>>>>>> [email protected]> > >>>> >>>>>>> > wrote: > >>>> >>>>>>> > > >>>> >>>>>>> >> > >>>> >>>>>>> >> > >>>> >>>>>>> >> On 2022/01/18 09:20:24 "张铎(Duo Zhang)" wrote: > >>>> >>>>>>> >> > Thank you Gavin. > >>>> >>>>>>> >> > > >>>> >>>>>>> >> > We have some urls in our jenkins files which references > the > >>>> >>>>>>> artifacts of > >>>> >>>>>>> >> > the jenkins jobs, so a simple copy or move can not work. > >>>> For > >>>> >>>>>>> example, > >>>> >>>>>>> >> here: > >>>> >>>>>>> >> > > >>>> >>>>>>> >> > > >>>> >>>>>>> >> > >>>> >>>>>>> > >>>> > https://github.com/apache/hbase/blob/c9bcd87b34a15d200a55ec7fdc2b1d86e3367a8c/dev-support/Jenkinsfile_GitHub#L47 > >>>> >>>>>>> >> > > >>>> >>>>>>> >> > So I prefer we migrate the jobs by ourselves. Will file a > >>>> HBase > >>>> >>>>>>> issue > >>>> >>>>>>> >> to do > >>>> >>>>>>> >> > this. > >>>> >>>>>>> >> > > >>>> >>>>>>> >> > And will report any problems on INFRA_22760 while > >>>> migrating. > >>>> >>>>>>> When the > >>>> >>>>>>> >> > migration is half done, you can move for example, 5 hbase > >>>> nodes > >>>> >>>>>>> from > >>>> >>>>>>> >> > ci-hadoop to ci-hbase, and once the migration is finally > >>>> done, > >>>> >>>>>>> you can > >>>> >>>>>>> >> move > >>>> >>>>>>> >> > all the remaining hbase nodes from ci-hadoop to ci-hbase. > >>>> >>>>>>> >> > > >>>> >>>>>>> >> > WDYT? > >>>> >>>>>>> >> > >>>> >>>>>>> >> However you want to do it is fine with me :) > >>>> >>>>>>> >> > >>>> >>>>>>> >> You have 10 nodes on there right now to get you started. > >>>> >>>>>>> >> > >>>> >>>>>>> >> Let me know how you get on > >>>> >>>>>>> >> > >>>> >>>>>>> >> > > >>>> >>>>>>> >> > Thanks. > >>>> >>>>>>> >> > > >>>> >>>>>>> >> > Gavin McDonald <[email protected]> 于2022年1月18日周二 > >>>> 16:16写道： > >>>> >>>>>>> >> > > >>>> >>>>>>> >> > > Hi All, > >>>> >>>>>>> >> > > > >>>> >>>>>>> >> > > I created an Infra ticket for tracking. > >>>> >>>>>>> >> > > > >>>> >>>>>>> >> > > https://issues.apache.org/jira/browse/INFRA-22760 > >>>> >>>>>>> >> > > > >>>> >>>>>>> >> > > For now, I left your current nodes hbase1 to hbase10 in > >>>> >>>>>>> ci-hadoop.a.o > >>>> >>>>>>> >> > > > >>>> >>>>>>> >> > > I have created for you 10 new nodes so far out of our > >>>> Hetzner > >>>> >>>>>>> DC > >>>> >>>>>>> >> > > of a similar specifications to the hbase1-10. > >>>> >>>>>>> >> > > > >>>> >>>>>>> >> > > Those are hbase11-20 and are available right now for > >>>> testing > >>>> >>>>>>> on > >>>> >>>>>>> >> > > ci-hbase.apache.org > >>>> >>>>>>> >> > > > >>>> >>>>>>> >> > > I do not know your workflow well enough to decide the > >>>> next > >>>> >>>>>>> steps for > >>>> >>>>>>> >> you > >>>> >>>>>>> >> > > for testing. > >>>> >>>>>>> >> > > > >>>> >>>>>>> >> > > Options include: > >>>> >>>>>>> >> > > > >>>> >>>>>>> >> > > 1. I can 'move' all jobs out of ci-hadoop onto ci-hbase > >>>> >>>>>>> >> > > 2. I can 'move' one or a number of jobs out of > ci-hadoop > >>>> onto > >>>> >>>>>>> ci-hbase > >>>> >>>>>>> >> > > 3. Similar to 1 or 2, but 'copy' any number of jobs > over > >>>> for > >>>> >>>>>>> testing. > >>>> >>>>>>> >> > > - I like copy, however quite a few projects jobs > >>>> would > >>>> >>>>>>> not be > >>>> >>>>>>> >> > > appropriate > >>>> >>>>>>> >> > > to have the same job running twice, including > >>>> those that > >>>> >>>>>>> >> deploy, or > >>>> >>>>>>> >> > > those that > >>>> >>>>>>> >> > > build from PRs and continue a workflow to report > >>>> back, > >>>> >>>>>>> or > >>>> >>>>>>> >> approve > >>>> >>>>>>> >> > > for merge. > >>>> >>>>>>> >> > > So for a copy, would need to be a job that just > >>>> 'builds > >>>> >>>>>>> and > >>>> >>>>>>> >> tests' > >>>> >>>>>>> >> > > with no outflow. > >>>> >>>>>>> >> > > 4. You guys create new jobs on ci-hbase.apache.org to > >>>> test > >>>> >>>>>>> the nodes > >>>> >>>>>>> >> > > setup and the > >>>> >>>>>>> >> > > main controller setup, without affecting current > >>>> jobs. > >>>> >>>>>>> >> > > > >>>> >>>>>>> >> > > Obviously, you may to tweak version controlled script > >>>> files > >>>> >>>>>>> such as a > >>>> >>>>>>> >> > > JenkinsFile, which makes the copy options harder unless > >>>> >>>>>>> testing from a > >>>> >>>>>>> >> > > branch created for the purpose. > >>>> >>>>>>> >> > > > >>>> >>>>>>> >> > > I may be overthinking things, and option 1 , the > >>>> simplest and > >>>> >>>>>>> most > >>>> >>>>>>> >> > > straight forward from my end would also suit you guys > >>>> and we > >>>> >>>>>>> deal > >>>> >>>>>>> >> with any > >>>> >>>>>>> >> > > breakage as it happens (I'm thinking of integrations > with > >>>> >>>>>>> other > >>>> >>>>>>> >> services, > >>>> >>>>>>> >> > > Credentials/Secrets, Plugins I haven't installed that > are > >>>> >>>>>>> needed etc.) > >>>> >>>>>>> >> > > > >>>> >>>>>>> >> > > Let me know please ASAP how you want to proceed so we > >>>> can get > >>>> >>>>>>> using > >>>> >>>>>>> >> these > >>>> >>>>>>> >> > > nodes, after which I can then proceed to move hbase1-10 > >>>> nodes > >>>> >>>>>>> over > >>>> >>>>>>> >> also. > >>>> >>>>>>> >> > > And then start adding more into the mix. > >>>> >>>>>>> >> > > > >>>> >>>>>>> >> > > Oh. The 'git-websites' label should be a no-op. It is > >>>> >>>>>>> available on > >>>> >>>>>>> >> > > ci-hbase.a.o as the nodes with that label are shared > and > >>>> >>>>>>> leased out to > >>>> >>>>>>> >> > > controllers as needed upon request of a job. > >>>> >>>>>>> >> > > > >>>> >>>>>>> >> > > Thanks > >>>> >>>>>>> >> > > > >>>> >>>>>>> >> > > Gav... > >>>> >>>>>>> >> > > > >>>> >>>>>>> >> > > > >>>> >>>>>>> >> > > On 2021/12/09 15:07:26 "张铎(Duo Zhang)" wrote: > >>>> >>>>>>> >> > > > Thanks you Gavin. > >>>> >>>>>>> >> > > > > >>>> >>>>>>> >> > > > You can do it in your convenience, just let me know > so > >>>> I > >>>> >>>>>>> could > >>>> >>>>>>> >> change the > >>>> >>>>>>> >> > > > label in the jenkinsfile of our jenkins jobs, at > least > >>>> our > >>>> >>>>>>> pre > >>>> >>>>>>> >> commit job > >>>> >>>>>>> >> > > > still uses the Hadoop label. > >>>> >>>>>>> >> > > > > >>>> >>>>>>> >> > > > And another thing is about the hbase_generate_website > >>>> job, > >>>> >>>>>>> it needs > >>>> >>>>>>> >> to be > >>>> >>>>>>> >> > > > executed on the nodes under the git-websites label. > >>>> I'm not > >>>> >>>>>>> sure if > >>>> >>>>>>> >> the > >>>> >>>>>>> >> > > > label is still available after the migration. > >>>> >>>>>>> >> > > > > >>>> >>>>>>> >> > > > Gavin McDonald <[email protected]> 于2021年12月8日周三 > >>>> >>>>>>> 17:28写道： > >>>> >>>>>>> >> > > > > >>>> >>>>>>> >> > > > > Hi HBase devs. > >>>> >>>>>>> >> > > > > > >>>> >>>>>>> >> > > > > This is to let you know that Infra has created a > >>>> >>>>>>> dedicated Jenkins > >>>> >>>>>>> >> > > > > Controller at https://ci-hbase.apache.org . > >>>> >>>>>>> >> > > > > > >>>> >>>>>>> >> > > > > You have new incoming donated agents which will get > >>>> >>>>>>> connected to > >>>> >>>>>>> >> this > >>>> >>>>>>> >> > > new > >>>> >>>>>>> >> > > > > controller by Infra. > >>>> >>>>>>> >> > > > > > >>>> >>>>>>> >> > > > > You also have 10 or so agents connected to > >>>> >>>>>>> >> > > https://ci-hadoop.apache.org - > >>>> >>>>>>> >> > > > > I > >>>> >>>>>>> >> > > > > would like to move these agents - and all of your > >>>> jobs - > >>>> >>>>>>> over to > >>>> >>>>>>> >> the > >>>> >>>>>>> >> > > new > >>>> >>>>>>> >> > > > > controller as soon as possible. > >>>> >>>>>>> >> > > > > > >>>> >>>>>>> >> > > > > You may or may not need to reconfigure your jobs > and > >>>> or > >>>> >>>>>>> >> JenkinsFile or > >>>> >>>>>>> >> > > > > other. > >>>> >>>>>>> >> > > > > > >>>> >>>>>>> >> > > > > Can I get the go ahead to move these at my > >>>> convenience or > >>>> >>>>>>> suggest > >>>> >>>>>>> >> > > please a > >>>> >>>>>>> >> > > > > timeline for Infra to move these agents. > >>>> >>>>>>> >> > > > > > >>>> >>>>>>> >> > > > > Thanks! > >>>> >>>>>>> >> > > > > > >>>> >>>>>>> >> > > > > -- > >>>> >>>>>>> >> > > > > > >>>> >>>>>>> >> > > > > *Gavin McDonald* > >>>> >>>>>>> >> > > > > Systems Administrator > >>>> >>>>>>> >> > > > > ASF Infrastructure Team > >>>> >>>>>>> >> > > > > > >>>> >>>>>>> >> > > > > >>>> >>>>>>> >> > > > >>>> >>>>>>> >> > > >>>> >>>>>>> >> > >>>> >>>>>>> > > >>>> >>>>>>> > >>>> >>>>>>> -- > >>>> >>>>>>> > >>>> >>>>>>> *Gavin McDonald* > >>>> >>>>>>> Systems Administrator > >>>> >>>>>>> ASF Infrastructure Team > >>>> >>>>>>> > >>>> >>>>>> > >>>> >>>>> > >>>> >>>>> -- > >>>> >>>>> > >>>> >>>>> *Gavin McDonald* > >>>> >>>>> Systems Administrator > >>>> >>>>> ASF Infrastructure Team > >>>> >>>>> > >>>> >>>> > >>>> > >>>> -- > >>>> > >>>> *Gavin McDonald* > >>>> Systems Administrator > >>>> ASF Infrastructure Team > >>>> > >>> > > -- > > *Gavin McDonald* > Systems Administrator > ASF Infrastructure Team >
