Thanks a lot Olaf, totally missed the credentials for AWS, I was able to get the new hostnames and ssh in. The master's root partition seems to be full afaics, and under /home/jenkins/jobs there is a ton of old build data (even from Bigtop 1.1.0 etc..). I am not very expert in jenkins but I am wondering if a Global build discarder (found in https://ci.bigtop.apache.org/configure) could be something worth testing. Was it done in the past? Is there any downside in using it? For example, we could keep the last 6 months (or any time window) worth of build data. Let me know what you think!
Luca On Sat, Dec 4, 2021 at 9:08 PM Olaf Flebbe <[email protected]> wrote: > > Hi Luca, > > the name changed. There is now some space available. > > Check with the EC2 console to see the current public IP adresses . > https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Instances:instanceState=running > > Best > Olaf > > > Am 04.12.2021 um 18:23 schrieb Luca Toscano <[email protected]>: > > > > Hi :) > > > > https://ci.bigtop.apache.org/log/all seems to indicate that we are > > again in trouble, docker-slave-02 seems full. I got the credentials to > > access the node but I can reach it (my ssh session hangs > > indefinitely), is there anybody that can take a look? > > > > Thanks, > > > > Luca > > > > On Sat, Nov 27, 2021 at 8:49 PM Olaf Flebbe <[email protected]> wrote: > >> > >> Hi, > >> > >> I had a quick look around. Job History was excessive: availlable back to > >> 2017, I deleted all until 2019 and set the maximum life time to one year. > >> I see that not any job from bigtopstore have been removed so far. Now > >> having 52995 jobs on disk. > >> I removed all bug the last ~3000 job dirs. > >> > >> Can someone please look at the config of „bigpetstore“, please? > >> > >> At least we have some air to breath now again on jenkins. > >> > >> Olaf > >> > >> > >>> Am 27.11.2021 um 20:04 schrieb Olaf Flebbe <[email protected]>: > >>> > >>> Regarding setup of jenkins, you can find a lot of info here: > >>> > >>> https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide > >>> <https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide> > >>> > >>> Olaf > >>> > >>> > >>>> Am 27.11.2021 um 19:46 schrieb Olaf Flebbe <[email protected] > >>>> <mailto:[email protected]>>: > >>>> > >>>> I am working with luca to get him access .. > >>>> > >>>> Olaf > >>>> > >>>>> Am 27.11.2021 um 18:29 schrieb Luca Toscano <[email protected] > >>>>> <mailto:[email protected]>>: > >>>>> > >>>>> Keep reporting issues, sorry :) > >>>>> > >>>>> I see from https://ci.bigtop.apache.org/log/all > >>>>> <https://ci.bigtop.apache.org/log/all> that we are having > >>>>> disk space issues on the Jenkins node, builds are stopped for the > >>>>> moment. No idea if there is a guide/tutorial for the cleanup, but I > >>>>> guess it means accessing to the VM/hosts that are running Jenkins and > >>>>> I have never done it (not even sure where to put my ssh key etc, what > >>>>> is the hostname, etc..). > >>>>> > >>>>> Thanks! > >>>>> > >>>>> Luca > >>>>> > >>>>> On Fri, Nov 26, 2021 at 7:10 PM Olaf Flebbe <[email protected] > >>>>> <mailto:[email protected]>> wrote: > >>>>>> > >>>>>> Hi Luca, > >>>>>> > >>>>>> you are right, 201 should be valid. However there seems to be some > >>>>>> compatibility problem with at least this repo. So comment it out for > >>>>>> now if it resolves the isse. Or remove the whole proxy stuff > >>>>>> altoghether. We had stability issues with the internet connectivity at > >>>>>> that time. The proxy resolved that mostly. > >>>>>> > >>>>>> Olaf > >>>>>> > >>>>>> > >>>>>>> Am 26.11.2021 um 19:06 schrieb Luca Toscano <[email protected] > >>>>>>> <mailto:[email protected]>>: > >>>>>>> > >>>>>>> Hi Olaf, > >>>>>>> > >>>>>>> I see that https://repository.jboss.org/nexus/content/groups/public/ > >>>>>>> <https://repository.jboss.org/nexus/content/groups/public/> > >>>>>>> is valid though (http redirects to https), but I don't get why we > >>>>>>> should delete the line (also the code returns a HTTP 201 afaics so it > >>>>>>> seems not failing to find it). > >>>>>>> > >>>>>>> Lemme know, I am probably missing something, too many new things :) > >>>>>>> > >>>>>>> Luca > >>>>>>> > >>>>>>> On Fri, Nov 26, 2021 at 6:24 PM Olaf Flebbe <[email protected] > >>>>>>> <mailto:[email protected]>> wrote: > >>>>>>>> > >>>>>>>> Hi Luca, > >>>>>>>> > >>>>>>>> Seems like jboss.org <http://jboss.org/> <http://jboss.org/ > >>>>>>>> <http://jboss.org/>> repository does not exist any more. > >>>>>>>> > >>>>>>>> Try to delete this line: > >>>>>>>> https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 > >>>>>>>> > >>>>>>>> <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462> > >>>>>>>> > >>>>>>>> <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 > >>>>>>>> > >>>>>>>> <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462>> > >>>>>>>> > >>>>>>>> Olaf > >>>>>>>> > >>>>>>>> > >>>>>>>>> Am 26.11.2021 um 14:23 schrieb Luca Toscano <[email protected] > >>>>>>>>> <mailto:[email protected]>>: > >>>>>>>>> > >>>>>>>>> Hi Olaf, > >>>>>>>>> > >>>>>>>>> I restarted the jobs and some packages started to build, thanks! I > >>>>>>>>> found another weird error though: > >>>>>>>>> > >>>>>>>>> """ > >>>>>>>>> * What went wrong: > >>>>>>>>> Execution failed for task ':configure-nexus-repository.jboss.org > >>>>>>>>> <http://configure-nexus-repository.jboss.org/>'. > >>>>>>>>>> Failed to configure Nexus proxy repository.jboss.org > >>>>>>>>>> <http://repository.jboss.org/> with http code 201 returned. Run > >>>>>>>>>> with --info option to see the executed command > >>>>>>>>> """ > >>>>>>>>> More info: > >>>>>>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console > >>>>>>>>> > >>>>>>>>> <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console> > >>>>>>>>> > >>>>>>>>> I am wondering if https://github.com/apache/bigtop/pull/834 > >>>>>>>>> <https://github.com/apache/bigtop/pull/834> could help :) > >>>>>>>>> > >>>>>>>>> Luca > >>>>>>>>> > >>>>>>>>> On Fri, Nov 26, 2021 at 11:44 AM Olaf Flebbe <[email protected] > >>>>>>>>> <mailto:[email protected]>> wrote: > >>>>>>>>>> > >>>>>>>>>> Hi Luca, > >>>>>>>>>> > >>>>>>>>>> I once added a nexus container as an download cache for maven > >>>>>>>>>> dependencies, attached to a seperate docker network. It has to be > >>>>>>>>>> started manually if it failed. > >>>>>>>>>> https://ci.bigtop.apache.org/job/nexus-restart/ > >>>>>>>>>> <https://ci.bigtop.apache.org/job/nexus-restart/> > >>>>>>>>>> <https://ci.bigtop.apache.org/job/nexus-restart/ > >>>>>>>>>> <https://ci.bigtop.apache.org/job/nexus-restart/>> > >>>>>>>>>> > >>>>>>>>>> If this network is not present you might get this kind of error > >>>>>>>>>> So try to restart nexus. > >>>>>>>>>> > >>>>>>>>>> Best > >>>>>>>>>> Olaf > >>>>>>>>>> > >>>>>>>>>>> Am 26.11.2021 um 09:04 schrieb Luca Toscano > >>>>>>>>>>> <[email protected] <mailto:[email protected]>>: > >>>>>>>>>>> > >>>>>>>>>>> Update: > >>>>>>>>>>> > >>>>>>>>>>> - Found the config diff changes in Jenkins, so I know how to > >>>>>>>>>>> re-enable > >>>>>>>>>>> periodic execution. > >>>>>>>>>>> > >>>>>>>>>>> - Tried to kick off a build, but I stopped it since all the > >>>>>>>>>>> package > >>>>>>>>>>> builds were failing for the same error: > >>>>>>>>>>> ++ docker run -d --net=container:nexus > >>>>>>>>>>> bigtop/slaves:trunk-debian-10 /sbin/init > >>>>>>>>>>> docker: Error response from daemon: cannot join network of a non > >>>>>>>>>>> running container > >>>>>>>>>>> Example:https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console > >>>>>>>>>>> > >>>>>>>>>>> <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console> > >>>>>>>>>>> > >>>>>>>>>>> I have probably missed something, I don't exactly understand the > >>>>>>>>>>> error > >>>>>>>>>>> so any help would be appreciated :) > >>>>>>>>>>> > >>>>>>>>>>> Luca > >>>>>>>>>>> > >>>>>>>>>>> On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano > >>>>>>>>>>> <[email protected] <mailto:[email protected]>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>> Update: I added debian-11 support, if nobody disagrees I'd kick > >>>>>>>>>>>> off a > >>>>>>>>>>>> package build. I can also enable periodic execution but I am not > >>>>>>>>>>>> sure > >>>>>>>>>>>> what I should put in the related time interval field (I didn't > >>>>>>>>>>>> find it > >>>>>>>>>>>> in the docs). > >>>>>>>>>>>> > >>>>>>>>>>>> Luca > >>>>>>>>>>>> > >>>>>>>>>>>> On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano > >>>>>>>>>>>> <[email protected] <mailto:[email protected]>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> Hi Olaf! Thanks a lot :) > >>>>>>>>>>>>> > >>>>>>>>>>>>> I managed to change CI to allow Debian 11 (minus a little error > >>>>>>>>>>>>> for > >>>>>>>>>>>>> Ubuntu that seems unrelated, see > >>>>>>>>>>>>> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/ > >>>>>>>>>>>>> > >>>>>>>>>>>>> <https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/>) > >>>>>>>>>>>>> and I was able to build the hadoop packages locally via the new > >>>>>>>>>>>>> Debian > >>>>>>>>>>>>> 11 images published to Dockerhub. > >>>>>>>>>>>>> > >>>>>>>>>>>>> IIUC the last step is to modify > >>>>>>>>>>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/ > >>>>>>>>>>>>> > >>>>>>>>>>>>> <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/> > >>>>>>>>>>>>> and kick off a build, but I see "2021-10-02: temporarily disable > >>>>>>>>>>>>> periodic execution for preparing 3.0.0 release." so before doing > >>>>>>>>>>>>> anything I'd like some feedback (to avoid causing problems > >>>>>>>>>>>>> etc..). > >>>>>>>>>>>>> > >>>>>>>>>>>>> Luca > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe <[email protected]> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Hi Luca, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> I gave you super powers. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Best > >>>>>>>>>>>>>> Olaf > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Am 19.11.2021 um 08:15 schrieb Luca Toscano > >>>>>>>>>>>>>>> <[email protected]>: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hi everybody! > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I am working on > >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/BIGTOP-3600 and > >>>>>>>>>>>>>>> IIUC I'd need to add the CI jobs now, but of course I don't > >>>>>>>>>>>>>>> have > >>>>>>>>>>>>>>> permissions. If anybody has some time to help I'd be happy to > >>>>>>>>>>>>>>> learn > >>>>>>>>>>>>>>> the process and help out :) > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thanks! > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Luca > >>>>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>> > >>>> > >>> > >> >
