Update: I have deleted old build files from the Jenkins UI, mostly related to old Bigtop releases (1.4/1.5), keeping only the last 3. The Jenkins master has now ~470GB of free disk space, that is more than enough to keep building the current projects without incurring in disk space issues.
Question: should we delete all old builds, keeping only data for the projects that are actually being used (like Bigtop 3.x, trunk, etc..) ? Is there any value in keeping logs/etc.. about what has been done in the past? What I mean is if we have use cases that need us to go back in time to inspect what was done :) Thanks! Luca On Mon, Dec 6, 2021 at 9:57 PM Luca Toscano <[email protected]> wrote: > > Update: I have deleted (via the UI) the failed builds that I tried in > these days for Trunk packages, and I freed ~100G. Is there any > specific reason to keep more than 2/3 builds for each project by > default? > > The idea would be to do something like: > https://plugins.jenkins.io/build-discarder/#plugin-content-getting-started > > My understanding is that old builds are kept for no real reason, all > packages are stored in the Apache archive and we don't really need to > keep build logs/config around. If nobody opposes it, I'd add a basic > global config to keep 3 builds for each project and discard the rest. > > Let me know what you think! > > Luca > > > Luca > > On Sun, Dec 5, 2021 at 10:02 AM Luca Toscano <[email protected]> wrote: > > > > Thanks a lot Olaf, totally missed the credentials for AWS, I was able > > to get the new hostnames and ssh in. The master's root partition seems > > to be full afaics, and under /home/jenkins/jobs there is a ton of old > > build data (even from Bigtop 1.1.0 etc..). I am not very expert in > > jenkins but I am wondering if a Global build discarder (found in > > https://ci.bigtop.apache.org/configure) could be something worth > > testing. Was it done in the past? Is there any downside in using it? > > For example, we could keep the last 6 months (or any time window) > > worth of build data. Let me know what you think! > > > > Luca > > > > On Sat, Dec 4, 2021 at 9:08 PM Olaf Flebbe <[email protected]> wrote: > > > > > > Hi Luca, > > > > > > the name changed. There is now some space available. > > > > > > Check with the EC2 console to see the current public IP adresses . > > > https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Instances:instanceState=running > > > > > > Best > > > Olaf > > > > > > > Am 04.12.2021 um 18:23 schrieb Luca Toscano <[email protected]>: > > > > > > > > Hi :) > > > > > > > > https://ci.bigtop.apache.org/log/all seems to indicate that we are > > > > again in trouble, docker-slave-02 seems full. I got the credentials to > > > > access the node but I can reach it (my ssh session hangs > > > > indefinitely), is there anybody that can take a look? > > > > > > > > Thanks, > > > > > > > > Luca > > > > > > > > On Sat, Nov 27, 2021 at 8:49 PM Olaf Flebbe <[email protected]> wrote: > > > >> > > > >> Hi, > > > >> > > > >> I had a quick look around. Job History was excessive: availlable back > > > >> to 2017, I deleted all until 2019 and set the maximum life time to one > > > >> year. > > > >> I see that not any job from bigtopstore have been removed so far. Now > > > >> having 52995 jobs on disk. > > > >> I removed all bug the last ~3000 job dirs. > > > >> > > > >> Can someone please look at the config of „bigpetstore“, please? > > > >> > > > >> At least we have some air to breath now again on jenkins. > > > >> > > > >> Olaf > > > >> > > > >> > > > >>> Am 27.11.2021 um 20:04 schrieb Olaf Flebbe <[email protected]>: > > > >>> > > > >>> Regarding setup of jenkins, you can find a lot of info here: > > > >>> > > > >>> https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide > > > >>> > > > >>> <https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide> > > > >>> > > > >>> Olaf > > > >>> > > > >>> > > > >>>> Am 27.11.2021 um 19:46 schrieb Olaf Flebbe <[email protected] > > > >>>> <mailto:[email protected]>>: > > > >>>> > > > >>>> I am working with luca to get him access .. > > > >>>> > > > >>>> Olaf > > > >>>> > > > >>>>> Am 27.11.2021 um 18:29 schrieb Luca Toscano <[email protected] > > > >>>>> <mailto:[email protected]>>: > > > >>>>> > > > >>>>> Keep reporting issues, sorry :) > > > >>>>> > > > >>>>> I see from https://ci.bigtop.apache.org/log/all > > > >>>>> <https://ci.bigtop.apache.org/log/all> that we are having > > > >>>>> disk space issues on the Jenkins node, builds are stopped for the > > > >>>>> moment. No idea if there is a guide/tutorial for the cleanup, but I > > > >>>>> guess it means accessing to the VM/hosts that are running Jenkins > > > >>>>> and > > > >>>>> I have never done it (not even sure where to put my ssh key etc, > > > >>>>> what > > > >>>>> is the hostname, etc..). > > > >>>>> > > > >>>>> Thanks! > > > >>>>> > > > >>>>> Luca > > > >>>>> > > > >>>>> On Fri, Nov 26, 2021 at 7:10 PM Olaf Flebbe <[email protected] > > > >>>>> <mailto:[email protected]>> wrote: > > > >>>>>> > > > >>>>>> Hi Luca, > > > >>>>>> > > > >>>>>> you are right, 201 should be valid. However there seems to be some > > > >>>>>> compatibility problem with at least this repo. So comment it out > > > >>>>>> for now if it resolves the isse. Or remove the whole proxy stuff > > > >>>>>> altoghether. We had stability issues with the internet > > > >>>>>> connectivity at that time. The proxy resolved that mostly. > > > >>>>>> > > > >>>>>> Olaf > > > >>>>>> > > > >>>>>> > > > >>>>>>> Am 26.11.2021 um 19:06 schrieb Luca Toscano > > > >>>>>>> <[email protected] <mailto:[email protected]>>: > > > >>>>>>> > > > >>>>>>> Hi Olaf, > > > >>>>>>> > > > >>>>>>> I see that > > > >>>>>>> https://repository.jboss.org/nexus/content/groups/public/ > > > >>>>>>> <https://repository.jboss.org/nexus/content/groups/public/> > > > >>>>>>> is valid though (http redirects to https), but I don't get why we > > > >>>>>>> should delete the line (also the code returns a HTTP 201 afaics > > > >>>>>>> so it > > > >>>>>>> seems not failing to find it). > > > >>>>>>> > > > >>>>>>> Lemme know, I am probably missing something, too many new things > > > >>>>>>> :) > > > >>>>>>> > > > >>>>>>> Luca > > > >>>>>>> > > > >>>>>>> On Fri, Nov 26, 2021 at 6:24 PM Olaf Flebbe <[email protected] > > > >>>>>>> <mailto:[email protected]>> wrote: > > > >>>>>>>> > > > >>>>>>>> Hi Luca, > > > >>>>>>>> > > > >>>>>>>> Seems like jboss.org <http://jboss.org/> <http://jboss.org/ > > > >>>>>>>> <http://jboss.org/>> repository does not exist any more. > > > >>>>>>>> > > > >>>>>>>> Try to delete this line: > > > >>>>>>>> https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 > > > >>>>>>>> > > > >>>>>>>> <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462> > > > >>>>>>>> > > > >>>>>>>> <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462 > > > >>>>>>>> > > > >>>>>>>> <https://github.com/apache/bigtop/blob/5f863aae467198070c1230558446a19f308a66ae/build.gradle#L462>> > > > >>>>>>>> > > > >>>>>>>> Olaf > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>>> Am 26.11.2021 um 14:23 schrieb Luca Toscano > > > >>>>>>>>> <[email protected] <mailto:[email protected]>>: > > > >>>>>>>>> > > > >>>>>>>>> Hi Olaf, > > > >>>>>>>>> > > > >>>>>>>>> I restarted the jobs and some packages started to build, > > > >>>>>>>>> thanks! I > > > >>>>>>>>> found another weird error though: > > > >>>>>>>>> > > > >>>>>>>>> """ > > > >>>>>>>>> * What went wrong: > > > >>>>>>>>> Execution failed for task > > > >>>>>>>>> ':configure-nexus-repository.jboss.org > > > >>>>>>>>> <http://configure-nexus-repository.jboss.org/>'. > > > >>>>>>>>>> Failed to configure Nexus proxy repository.jboss.org > > > >>>>>>>>>> <http://repository.jboss.org/> with http code 201 returned. > > > >>>>>>>>>> Run with --info option to see the executed command > > > >>>>>>>>> """ > > > >>>>>>>>> More info: > > > >>>>>>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console > > > >>>>>>>>> > > > >>>>>>>>> <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/745/COMPONENTS=flink,OS=debian-9/console> > > > >>>>>>>>> > > > >>>>>>>>> I am wondering if https://github.com/apache/bigtop/pull/834 > > > >>>>>>>>> <https://github.com/apache/bigtop/pull/834> could help :) > > > >>>>>>>>> > > > >>>>>>>>> Luca > > > >>>>>>>>> > > > >>>>>>>>> On Fri, Nov 26, 2021 at 11:44 AM Olaf Flebbe <[email protected] > > > >>>>>>>>> <mailto:[email protected]>> wrote: > > > >>>>>>>>>> > > > >>>>>>>>>> Hi Luca, > > > >>>>>>>>>> > > > >>>>>>>>>> I once added a nexus container as an download cache for maven > > > >>>>>>>>>> dependencies, attached to a seperate docker network. It has to > > > >>>>>>>>>> be started manually if it failed. > > > >>>>>>>>>> https://ci.bigtop.apache.org/job/nexus-restart/ > > > >>>>>>>>>> <https://ci.bigtop.apache.org/job/nexus-restart/> > > > >>>>>>>>>> <https://ci.bigtop.apache.org/job/nexus-restart/ > > > >>>>>>>>>> <https://ci.bigtop.apache.org/job/nexus-restart/>> > > > >>>>>>>>>> > > > >>>>>>>>>> If this network is not present you might get this kind of error > > > >>>>>>>>>> So try to restart nexus. > > > >>>>>>>>>> > > > >>>>>>>>>> Best > > > >>>>>>>>>> Olaf > > > >>>>>>>>>> > > > >>>>>>>>>>> Am 26.11.2021 um 09:04 schrieb Luca Toscano > > > >>>>>>>>>>> <[email protected] <mailto:[email protected]>>: > > > >>>>>>>>>>> > > > >>>>>>>>>>> Update: > > > >>>>>>>>>>> > > > >>>>>>>>>>> - Found the config diff changes in Jenkins, so I know how to > > > >>>>>>>>>>> re-enable > > > >>>>>>>>>>> periodic execution. > > > >>>>>>>>>>> > > > >>>>>>>>>>> - Tried to kick off a build, but I stopped it since all the > > > >>>>>>>>>>> package > > > >>>>>>>>>>> builds were failing for the same error: > > > >>>>>>>>>>> ++ docker run -d --net=container:nexus > > > >>>>>>>>>>> bigtop/slaves:trunk-debian-10 /sbin/init > > > >>>>>>>>>>> docker: Error response from daemon: cannot join network of a > > > >>>>>>>>>>> non > > > >>>>>>>>>>> running container > > > >>>>>>>>>>> Example:https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console > > > >>>>>>>>>>> > > > >>>>>>>>>>> <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=solr,OS=debian-10/lastBuild/console> > > > >>>>>>>>>>> > > > >>>>>>>>>>> I have probably missed something, I don't exactly understand > > > >>>>>>>>>>> the error > > > >>>>>>>>>>> so any help would be appreciated :) > > > >>>>>>>>>>> > > > >>>>>>>>>>> Luca > > > >>>>>>>>>>> > > > >>>>>>>>>>> On Thu, Nov 25, 2021 at 12:27 PM Luca Toscano > > > >>>>>>>>>>> <[email protected] <mailto:[email protected]>> > > > >>>>>>>>>>> wrote: > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Update: I added debian-11 support, if nobody disagrees I'd > > > >>>>>>>>>>>> kick off a > > > >>>>>>>>>>>> package build. I can also enable periodic execution but I am > > > >>>>>>>>>>>> not sure > > > >>>>>>>>>>>> what I should put in the related time interval field (I > > > >>>>>>>>>>>> didn't find it > > > >>>>>>>>>>>> in the docs). > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Luca > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> On Sun, Nov 21, 2021 at 7:10 PM Luca Toscano > > > >>>>>>>>>>>> <[email protected] <mailto:[email protected]>> > > > >>>>>>>>>>>> wrote: > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> Hi Olaf! Thanks a lot :) > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> I managed to change CI to allow Debian 11 (minus a little > > > >>>>>>>>>>>>> error for > > > >>>>>>>>>>>>> Ubuntu that seems unrelated, see > > > >>>>>>>>>>>>> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/ > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> <https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/>) > > > >>>>>>>>>>>>> and I was able to build the hadoop packages locally via the > > > >>>>>>>>>>>>> new Debian > > > >>>>>>>>>>>>> 11 images published to Dockerhub. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> IIUC the last step is to modify > > > >>>>>>>>>>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/ > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> <https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/> > > > >>>>>>>>>>>>> and kick off a build, but I see "2021-10-02: temporarily > > > >>>>>>>>>>>>> disable > > > >>>>>>>>>>>>> periodic execution for preparing 3.0.0 release." so before > > > >>>>>>>>>>>>> doing > > > >>>>>>>>>>>>> anything I'd like some feedback (to avoid causing problems > > > >>>>>>>>>>>>> etc..). > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> Luca > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> On Fri, Nov 19, 2021 at 8:53 PM Olaf Flebbe > > > >>>>>>>>>>>>> <[email protected]> wrote: > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> Hi Luca, > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> I gave you super powers. > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>> Best > > > >>>>>>>>>>>>>> Olaf > > > >>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> Am 19.11.2021 um 08:15 schrieb Luca Toscano > > > >>>>>>>>>>>>>>> <[email protected]>: > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> Hi everybody! > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> I am working on > > > >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/BIGTOP-3600 and > > > >>>>>>>>>>>>>>> IIUC I'd need to add the CI jobs now, but of course I > > > >>>>>>>>>>>>>>> don't have > > > >>>>>>>>>>>>>>> permissions. If anybody has some time to help I'd be > > > >>>>>>>>>>>>>>> happy to learn > > > >>>>>>>>>>>>>>> the process and help out :) > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> Thanks! > > > >>>>>>>>>>>>>>> > > > >>>>>>>>>>>>>>> Luca > > > >>>>>>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>> > > > >>>>>> > > > >>>> > > > >>> > > > >> > > >
