As of a few days ago. the Publish over SSH plugin was made unavailable due to a CVE. Looking into it but hopefully a simple fix that will be rolled out soon
On Sat, Jan 22, 2022 at 10:06 AM 张铎(Duo Zhang) <palomino...@gmail.com> wrote: > Ah, another problem... > > java.lang.NoSuchMethodError: No such DSL method 'sshTransfer' found among > steps > > > Seems on the new jenkins we do not have the support for transfering data > to nightlies... > > 张铎(Duo Zhang) <palomino...@gmail.com> 于2022年1月22日周六 17:04写道： > >> Thank you Gavin. The builds are started now. >> >> Let's wait for the result. >> >> And on installing plugins, I haven't seen any system management related >> button on the https://ci-hbase.apache.org/ page... >> On the https://jenkins-ccos.apache.org/job/masters/job/HBase/ page, >> click the 'Config Files' button, there is a 'Manage Jenkins' button but I >> have no permission to click it... >> >> Another thing is hbase20 is offline because of readonly filesystem. PTAL. >> >> Thanks a lot. >> >> Gavin McDonald <gmcdon...@apache.org> 于2022年1月22日周六 16:42写道： >> >>> Hi, >>> Timestamper plugin installed. >>> >>> By the way I thought I had given the PMC members permissions to manage >>> Jenkins >>> on ci-hbase including installing plugins etc, can someone check? >>> >>> >>> On Sat, Jan 22, 2022 at 7:34 AM 张铎(Duo Zhang) <palomino...@gmail.com> >>> wrote: >>> >>>> We faced a strange issue.. >>>> >>>> rg.codehaus.groovy.control.MultipleCompilationErrorsException: startup >>>> failed: >>>> WorkflowScript: 29: Invalid option type "timestamps". Valid option types: >>>> [buildDiscarder, catchError, checkoutToSubdirectory, >>>> disableConcurrentBuilds, disableResume, durabilityHint, >>>> githubProjectProperty, newContainerPerStage, overrideIndexTriggers, >>>> parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds, >>>> retry, script, skipDefaultCheckout, skipStagesAfterUnstable, timeout, >>>> waitUntil, warnError, withChecks, withContext, withCredentials, withEnv, >>>> wrap, ws] @ line 29, column 5. >>>> timestamps() >>>> ^ >>>> >>>> 1 error >>>> >>>> at >>>> org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310) >>>> at >>>> org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085) >>>> at >>>> org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603) >>>> at >>>> org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581) >>>> at >>>> org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558) >>>> at >>>> groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298) >>>> at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268) >>>> at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688) >>>> at groovy.lang.GroovyShell.parse(GroovyShell.java:700) >>>> at >>>> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142) >>>> at >>>> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127) >>>> at >>>> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:571) >>>> at >>>> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:523) >>>> at >>>> org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:334) >>>> at hudson.model.ResourceController.execute(ResourceController.java:97) >>>> at hudson.model.Executor.run(Executor.java:429) >>>> >>>> >>>> After googling, it seems a restart of jenkins can fix it, or we do not >>>> have timestamper plugin installed? Mind taking a look Gavin? >>>> >>>> Thanks. >>>> >>>> 张铎(Duo Zhang) <palomino...@gmail.com> 于2022年1月22日周六 11:40写道： >>>> >>>>> Oh, just saw that we could do this by ourselves. >>>>> >>>>> Let me have a try. >>>>> >>>>> 张铎(Duo Zhang) <palomino...@gmail.com> 于2022年1月22日周六 11:18写道： >>>>> >>>>>> OK, so while copying you could keep the old jenkins builds? >>>>>> That's awesome. >>>>>> >>>>>> Then please help 'copying' HBase-Find-Flaky-Tests >>>>>> and HBase-Flaky-Tests to ci-hbase too? We need the history builds to >>>>>> calucate the flaky test list so if all the old build data can be kept >>>>>> then >>>>>> it will help us a lot. >>>>>> >>>>>> Thanks~ >>>>>> >>>>>> 张铎(Duo Zhang) <palomino...@gmail.com> 于2022年1月22日周六 11:03写道： >>>>>> >>>>>>> Thank you Gavin. >>>>>>> >>>>>>> Let me first move some light weighted job first to see if everything >>>>>>> is OK. >>>>>>> >>>>>>> HBase Nightly, some flaky related jobs, and also the main pre commit >>>>>>> job for HBase are tied together. We need to move them at once. >>>>>>> >>>>>>> Gavin McDonald <gmcdon...@apache.org> 于2022年1月22日周六 02:44写道： >>>>>>> >>>>>>>> Thanks for your patience >>>>>>>> >>>>>>>> I have 'copied' over a job - >>>>>>>> https://ci-hbase.apache.org/job/HBase%20Nightly/ >>>>>>>> >>>>>>>> which of course made me aware of what plugins I needed to install. >>>>>>>> >>>>>>>> Feel free to delete that, or use it and delete the old, and let me >>>>>>>> know if have any issues >>>>>>>> with any other missing plugins. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Jan 20, 2022 at 1:11 PM 张铎(Duo Zhang) < >>>>>>>> palomino...@gmail.com> wrote: >>>>>>>> >>>>>>>>> I’ve also replied on the jira issue. >>>>>>>>> >>>>>>>>> You can try to copy a job to the new Jenkins. >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> >>>>>>>>> Gavin McDonald <gmcdon...@apache.org>于2022年1月20日 周四18:45写道： >>>>>>>>> >>>>>>>>>> Im taking a look ... >>>>>>>>>> >>>>>>>>>> On Thu, Jan 20, 2022 at 6:54 AM Sean Busbey <bus...@apache.org> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> > Hi Gavin >>>>>>>>>> > >>>>>>>>>> > I went to start migrating jobs to the ci-hbase controller, but >>>>>>>>>> the new job >>>>>>>>>> > workflow doesn't include pipelines or multibranch pipelines. We >>>>>>>>>> use both of >>>>>>>>>> > these job types for nearly all of our work. Can they be enabled? >>>>>>>>>> > >>>>>>>>>> > On Tue, Jan 18, 2022 at 5:04 AM Gavin McDonald < >>>>>>>>>> gmcdon...@apache.org> >>>>>>>>>> > wrote: >>>>>>>>>> > >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> On 2022/01/18 09:20:24 "张铎(Duo Zhang)" wrote: >>>>>>>>>> >> > Thank you Gavin. >>>>>>>>>> >> > >>>>>>>>>> >> > We have some urls in our jenkins files which references the >>>>>>>>>> artifacts of >>>>>>>>>> >> > the jenkins jobs, so a simple copy or move can not work. For >>>>>>>>>> example, >>>>>>>>>> >> here: >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> >> >>>>>>>>>> https://github.com/apache/hbase/blob/c9bcd87b34a15d200a55ec7fdc2b1d86e3367a8c/dev-support/Jenkinsfile_GitHub#L47 >>>>>>>>>> >> > >>>>>>>>>> >> > So I prefer we migrate the jobs by ourselves. Will file a >>>>>>>>>> HBase issue >>>>>>>>>> >> to do >>>>>>>>>> >> > this. >>>>>>>>>> >> > >>>>>>>>>> >> > And will report any problems on INFRA_22760 while migrating. >>>>>>>>>> When the >>>>>>>>>> >> > migration is half done, you can move for example, 5 hbase >>>>>>>>>> nodes from >>>>>>>>>> >> > ci-hadoop to ci-hbase, and once the migration is finally >>>>>>>>>> done, you can >>>>>>>>>> >> move >>>>>>>>>> >> > all the remaining hbase nodes from ci-hadoop to ci-hbase. >>>>>>>>>> >> > >>>>>>>>>> >> > WDYT? >>>>>>>>>> >> >>>>>>>>>> >> However you want to do it is fine with me :) >>>>>>>>>> >> >>>>>>>>>> >> You have 10 nodes on there right now to get you started. >>>>>>>>>> >> >>>>>>>>>> >> Let me know how you get on >>>>>>>>>> >> >>>>>>>>>> >> > >>>>>>>>>> >> > Thanks. >>>>>>>>>> >> > >>>>>>>>>> >> > Gavin McDonald <gmcdon...@apache.org> 于2022年1月18日周二 16:16写道： >>>>>>>>>> >> > >>>>>>>>>> >> > > Hi All, >>>>>>>>>> >> > > >>>>>>>>>> >> > > I created an Infra ticket for tracking. >>>>>>>>>> >> > > >>>>>>>>>> >> > > https://issues.apache.org/jira/browse/INFRA-22760 >>>>>>>>>> >> > > >>>>>>>>>> >> > > For now, I left your current nodes hbase1 to hbase10 in >>>>>>>>>> ci-hadoop.a.o >>>>>>>>>> >> > > >>>>>>>>>> >> > > I have created for you 10 new nodes so far out of our >>>>>>>>>> Hetzner DC >>>>>>>>>> >> > > of a similar specifications to the hbase1-10. >>>>>>>>>> >> > > >>>>>>>>>> >> > > Those are hbase11-20 and are available right now for >>>>>>>>>> testing on >>>>>>>>>> >> > > ci-hbase.apache.org >>>>>>>>>> >> > > >>>>>>>>>> >> > > I do not know your workflow well enough to decide the next >>>>>>>>>> steps for >>>>>>>>>> >> you >>>>>>>>>> >> > > for testing. >>>>>>>>>> >> > > >>>>>>>>>> >> > > Options include: >>>>>>>>>> >> > > >>>>>>>>>> >> > > 1. I can 'move' all jobs out of ci-hadoop onto ci-hbase >>>>>>>>>> >> > > 2. I can 'move' one or a number of jobs out of ci-hadoop >>>>>>>>>> onto ci-hbase >>>>>>>>>> >> > > 3. Similar to 1 or 2, but 'copy' any number of jobs over >>>>>>>>>> for testing. >>>>>>>>>> >> > > - I like copy, however quite a few projects jobs would >>>>>>>>>> not be >>>>>>>>>> >> > > appropriate >>>>>>>>>> >> > > to have the same job running twice, including those >>>>>>>>>> that >>>>>>>>>> >> deploy, or >>>>>>>>>> >> > > those that >>>>>>>>>> >> > > build from PRs and continue a workflow to report >>>>>>>>>> back, or >>>>>>>>>> >> approve >>>>>>>>>> >> > > for merge. >>>>>>>>>> >> > > So for a copy, would need to be a job that just >>>>>>>>>> 'builds and >>>>>>>>>> >> tests' >>>>>>>>>> >> > > with no outflow. >>>>>>>>>> >> > > 4. You guys create new jobs on ci-hbase.apache.org to >>>>>>>>>> test the nodes >>>>>>>>>> >> > > setup and the >>>>>>>>>> >> > > main controller setup, without affecting current jobs. >>>>>>>>>> >> > > >>>>>>>>>> >> > > Obviously, you may to tweak version controlled script >>>>>>>>>> files such as a >>>>>>>>>> >> > > JenkinsFile, which makes the copy options harder unless >>>>>>>>>> testing from a >>>>>>>>>> >> > > branch created for the purpose. >>>>>>>>>> >> > > >>>>>>>>>> >> > > I may be overthinking things, and option 1 , the simplest >>>>>>>>>> and most >>>>>>>>>> >> > > straight forward from my end would also suit you guys and >>>>>>>>>> we deal >>>>>>>>>> >> with any >>>>>>>>>> >> > > breakage as it happens (I'm thinking of integrations with >>>>>>>>>> other >>>>>>>>>> >> services, >>>>>>>>>> >> > > Credentials/Secrets, Plugins I haven't installed that are >>>>>>>>>> needed etc.) >>>>>>>>>> >> > > >>>>>>>>>> >> > > Let me know please ASAP how you want to proceed so we can >>>>>>>>>> get using >>>>>>>>>> >> these >>>>>>>>>> >> > > nodes, after which I can then proceed to move hbase1-10 >>>>>>>>>> nodes over >>>>>>>>>> >> also. >>>>>>>>>> >> > > And then start adding more into the mix. >>>>>>>>>> >> > > >>>>>>>>>> >> > > Oh. The 'git-websites' label should be a no-op. It is >>>>>>>>>> available on >>>>>>>>>> >> > > ci-hbase.a.o as the nodes with that label are shared and >>>>>>>>>> leased out to >>>>>>>>>> >> > > controllers as needed upon request of a job. >>>>>>>>>> >> > > >>>>>>>>>> >> > > Thanks >>>>>>>>>> >> > > >>>>>>>>>> >> > > Gav... >>>>>>>>>> >> > > >>>>>>>>>> >> > > >>>>>>>>>> >> > > On 2021/12/09 15:07:26 "张铎(Duo Zhang)" wrote: >>>>>>>>>> >> > > > Thanks you Gavin. >>>>>>>>>> >> > > > >>>>>>>>>> >> > > > You can do it in your convenience, just let me know so I >>>>>>>>>> could >>>>>>>>>> >> change the >>>>>>>>>> >> > > > label in the jenkinsfile of our jenkins jobs, at least >>>>>>>>>> our pre >>>>>>>>>> >> commit job >>>>>>>>>> >> > > > still uses the Hadoop label. >>>>>>>>>> >> > > > >>>>>>>>>> >> > > > And another thing is about the hbase_generate_website >>>>>>>>>> job, it needs >>>>>>>>>> >> to be >>>>>>>>>> >> > > > executed on the nodes under the git-websites label. I'm >>>>>>>>>> not sure if >>>>>>>>>> >> the >>>>>>>>>> >> > > > label is still available after the migration. >>>>>>>>>> >> > > > >>>>>>>>>> >> > > > Gavin McDonald <gmcdon...@apache.org> 于2021年12月8日周三 >>>>>>>>>> 17:28写道： >>>>>>>>>> >> > > > >>>>>>>>>> >> > > > > Hi HBase devs. >>>>>>>>>> >> > > > > >>>>>>>>>> >> > > > > This is to let you know that Infra has created a >>>>>>>>>> dedicated Jenkins >>>>>>>>>> >> > > > > Controller at https://ci-hbase.apache.org . >>>>>>>>>> >> > > > > >>>>>>>>>> >> > > > > You have new incoming donated agents which will get >>>>>>>>>> connected to >>>>>>>>>> >> this >>>>>>>>>> >> > > new >>>>>>>>>> >> > > > > controller by Infra. >>>>>>>>>> >> > > > > >>>>>>>>>> >> > > > > You also have 10 or so agents connected to >>>>>>>>>> >> > > https://ci-hadoop.apache.org - >>>>>>>>>> >> > > > > I >>>>>>>>>> >> > > > > would like to move these agents - and all of your jobs >>>>>>>>>> - over to >>>>>>>>>> >> the >>>>>>>>>> >> > > new >>>>>>>>>> >> > > > > controller as soon as possible. >>>>>>>>>> >> > > > > >>>>>>>>>> >> > > > > You may or may not need to reconfigure your jobs and or >>>>>>>>>> >> JenkinsFile or >>>>>>>>>> >> > > > > other. >>>>>>>>>> >> > > > > >>>>>>>>>> >> > > > > Can I get the go ahead to move these at my convenience >>>>>>>>>> or suggest >>>>>>>>>> >> > > please a >>>>>>>>>> >> > > > > timeline for Infra to move these agents. >>>>>>>>>> >> > > > > >>>>>>>>>> >> > > > > Thanks! >>>>>>>>>> >> > > > > >>>>>>>>>> >> > > > > -- >>>>>>>>>> >> > > > > >>>>>>>>>> >> > > > > *Gavin McDonald* >>>>>>>>>> >> > > > > Systems Administrator >>>>>>>>>> >> > > > > ASF Infrastructure Team >>>>>>>>>> >> > > > > >>>>>>>>>> >> > > > >>>>>>>>>> >> > > >>>>>>>>>> >> > >>>>>>>>>> >> >>>>>>>>>> > >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> *Gavin McDonald* >>>>>>>>>> Systems Administrator >>>>>>>>>> ASF Infrastructure Team >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> *Gavin McDonald* >>>>>>>> Systems Administrator >>>>>>>> ASF Infrastructure Team >>>>>>>> >>>>>>> >>> >>> -- >>> >>> *Gavin McDonald* >>> Systems Administrator >>> ASF Infrastructure Team >>> >> -- *Gavin McDonald* Systems Administrator ASF Infrastructure Team
