[sysadmin/ci-management] /: Try to align the number of folders between seed jobs and normal CI jobs.
Git commit 707e016918c0174235b1dc19883620f96f363572 by Ben Cooksley. Committed on 07/04/2022 at 05:41. Pushed by bcooksley into branch 'master'. Try to align the number of folders between seed jobs and normal CI jobs. This only affects Windows (guess everywhere else the path is absolute while on Windows it is relative) despite all our other jobs having the same layout. CCMAIL: kde-frameworks-devel@kde.org M +8-4.gitlab-ci.yml https://invent.kde.org/sysadmin/ci-management/commit/707e016918c0174235b1dc19883620f96f363572 diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 96c828d..225ecf3 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -154,7 +154,8 @@ frameworks_windows_qt515: - .windows_qt515 script: - . ci-utilities/resources/setup-msvc-env.ps1 -- python -u ci-utilities/seed-package-registry.py --seed-file latest/frameworks.yml --platform Windows +- cd .. +- python -u ci-management/ci-utilities/seed-package-registry.py --seed-file ci-management/latest/frameworks.yml --platform Windows ## Frameworks 6 jobs frameworks_suse_tumbleweed_qt62: @@ -206,7 +207,8 @@ release_service_windows_qt515: - .windows_qt515 script: - . ci-utilities/resources/setup-msvc-env.ps1 -- python -u ci-utilities/seed-package-registry.py --seed-file latest/release-service.yml --platform Windows +- cd .. +- python -u ci-management/ci-utilities/seed-package-registry.py --seed-file ci-management/latest/release-service.yml --platform Windows ## Plasma jobs @@ -276,7 +278,8 @@ independent_release_windows_qt515: - .windows_qt515 script: - . ci-utilities/resources/setup-msvc-env.ps1 -- python -u ci-utilities/seed-package-registry.py --seed-file latest/independent-release.yml --platform Windows +- cd .. +- python -u ci-management/ci-utilities/seed-package-registry.py --seed-file ci-management/latest/independent-release.yml --platform Windows ## PIM @@ -311,4 +314,5 @@ pim_windows_qt515: - .windows_qt515 script: - . ci-utilities/resources/setup-msvc-env.ps1 -- python -u ci-utilities/seed-package-registry.py --seed-file latest/pim.yml --platform Windows +- cd .. +- python -u ci-management/ci-utilities/seed-package-registry.py --seed-file ci-management/latest/pim.yml --platform Windows
Re: CI jobs
On 7/22/20 10:56 PM Nicolas Fella wrote: > On 22.07.20 22:13, David Hurka wrote: > > @ Get:1 http://deb.debian.org/debian testing/main amd64 perl-modules [...] > > @ > > > > Looks terrifying. Do CI jobs fetch stuff from lokal mirrors in their > > virtual environment, or is it terrifying? > > That should be answered in https://whydoesaptnotusehttps.com Thanks, nice to know!
Re: CI jobs
On Thu, Jul 23, 2020 at 8:56 AM Nicolas Fella wrote: > > > On 22.07.20 22:13, David Hurka wrote: > > @ Get:1 http://deb.debian.org/debian testing/main amd64 perl-modules [...] > > @ > > > > Looks terrifying. Do CI jobs fetch stuff from lokal mirrors in their virtual > > environment, or is it terrifying? > > That should be answered in https://whydoesaptnotusehttps.com It should be noted that the http:// and https:// implementations in apt are entirely separate codebases, with the https:// variant not supporting redirects (which is extremely annoying) > > > Cheers > > Nico Cheers, Ben
Re: CI jobs
On 22.07.20 22:13, David Hurka wrote: @ Get:1 http://deb.debian.org/debian testing/main amd64 perl-modules [...] @ Looks terrifying. Do CI jobs fetch stuff from lokal mirrors in their virtual environment, or is it terrifying? That should be answered in https://whydoesaptnotusehttps.com Cheers Nico
Re: CI jobs
On Thu, Jul 23, 2020 at 8:33 AM David Hurka wrote: > > > Hi David, > > Hi Ben, > > > The official CI jobs at https://build.kde.org/ work in different ways > > depending on the type of build [...] Jobs running on Gitlab are for > > experimental purposes only and should never be relied upon for any purposes. > > Yes, I am talking about Gitlab CI jobs. I wasn’t aware of build.kde.org that > moment, sorry for being unprecise. :) > > >From my understanding of our current Gitlab CI setup, all of these are run using Docker jobs and therefore are freshly spawned for each build. Hence the need to download those packages every time. I'm afraid we don't provide a cache for this. This is something that will likely change when the official CI system is ported over to Gitlab (Phase 2 of the Gitlab migration). Cheers, Ben
Re: CI jobs
> Hi David, Hi Ben, > The official CI jobs at https://build.kde.org/ work in different ways > depending on the type of build [...] Jobs running on Gitlab are for > experimental purposes only and should never be relied upon for any purposes. Yes, I am talking about Gitlab CI jobs. I wasn’t aware of build.kde.org that moment, sorry for being unprecise. :)
Re: CI jobs
On Thu, Jul 23, 2020 at 8:13 AM David Hurka wrote: > > Hi everyone, Hi David, > > I’m interested in how the CI jobs work. I expected that they are virtual > machines, set up to build the respective commit of the respective repository. The official CI jobs at https://build.kde.org/ work in different ways depending on the type of build: - SUSE (Linux) and Android builds are performed inside a Docker image that is spawned newly for each run - Windows and FreeBSD use permanent, fixed virtual machines that are re-used from run to run (with some cleanup performed automatically) Jobs running on Gitlab are for experimental purposes only and should never be relied upon for any purposes. > > I looked at the command line output of some Okular CI jobs, and they don’t > appear to be set up to build something. For each run, they appear to install > everything needed from apt, then build the whole repository in every detail. > This is not how I usually build stuff at home; my system already has > everything installed, and I only need to rebuild what has changed. Is all that > computation needed? Seems like a waste of CI minutes to me. > > clang_format makes this output: > > @ Get:1 http://deb.debian.org/debian testing/main amd64 perl-modules [...] > @ > > Looks terrifying. Do CI jobs fetch stuff from lokal mirrors in their virtual > environment, or is it terrifying? This is not something the official CI builds at https://build.kde.org/ do, as those builds are performed without root access (and also use SUSE for Linux and Ubuntu for Android, so it can't be using Debian) Therefore the above log must be from an experimental Gitlab CI job. > > Cheers, David > > Cheers, Ben
Re: CI jobs
+sysadmin Nate On 7/22/20 2:13 PM, David Hurka wrote: Hi everyone, I’m interested in how the CI jobs work. I expected that they are virtual machines, set up to build the respective commit of the respective repository. I looked at the command line output of some Okular CI jobs, and they don’t appear to be set up to build something. For each run, they appear to install everything needed from apt, then build the whole repository in every detail. This is not how I usually build stuff at home; my system already has everything installed, and I only need to rebuild what has changed. Is all that computation needed? Seems like a waste of CI minutes to me. clang_format makes this output: @ Get:1 http://deb.debian.org/debian testing/main amd64 perl-modules [...] @ Looks terrifying. Do CI jobs fetch stuff from lokal mirrors in their virtual environment, or is it terrifying? Cheers, David
CI jobs
Hi everyone, I’m interested in how the CI jobs work. I expected that they are virtual machines, set up to build the respective commit of the respective repository. I looked at the command line output of some Okular CI jobs, and they don’t appear to be set up to build something. For each run, they appear to install everything needed from apt, then build the whole repository in every detail. This is not how I usually build stuff at home; my system already has everything installed, and I only need to rebuild what has changed. Is all that computation needed? Seems like a waste of CI minutes to me. clang_format makes this output: @ Get:1 http://deb.debian.org/debian testing/main amd64 perl-modules [...] @ Looks terrifying. Do CI jobs fetch stuff from lokal mirrors in their virtual environment, or is it terrifying? Cheers, David
