Re: [PATCH] .gitlab-ci.d/cirrus: Remove the netbsd and openbsd jobs
On Fri, Apr 26, 2024 at 02:47:20PM +0200, Thomas Huth wrote: > With regards to NetBSD and OpenBSD, this is not a step backward since these > gitlab jobs were never run anyway (they could only be triggered manually, > but hardly anybody did that AFAIK). > > If we want to have proper support for those OSes, I think somebody would > need to set up a custom runner on a beefy KVM-capable server somewhere where > we could run the "make vm-build-*bsd" commands. By the way, are Eldon's CI > runners still around? IIRC they were capable of running KVM ? My datacenter had a power outage recently, so I disable my runner, and haven't prioritized bringing it back up until now. I am glad to get this going again, I'll look at it this weekend. It is a dedicated machine (R610 or Cisco UCS C220, I don't quite recall) with I believe ~2X cores and 100+ GB memory. I hit a couple of minor hurdles when CI started utilizing kubernetes runners, so one question I had was whether I should install a lightweight kubernetes system and run the gitlab kubernetes CI operator or the normal gitlab runner service. There should also be plenty of space to build *bsd VM's. Do pre-existing upstream BSD images have an nocloud support so that we can build from a stable updated base? Sorry I'm not super familiar with the BSD ecosystems, but happy to try to fill in the gaps. Thanks, Eldon
Re: New container build error: mountpoint does not exit
On Wed, May 24, 2023 at 01:09:30PM -0700, Richard Henderson wrote: > Hi Eldon, > > New this morning are some odd failures in the container build stage, e.g > > https://gitlab.com/qemu-project/qemu/-/jobs/4345796216#L235 > > cgroups: cgroup mountpoint does not exist: unknown > > There are several such failures in that pipeline. I've not seen this before, > nor was it > happening yesterday. > > Any ideas? Hi Richard, If I had to guess, it's probably related to some changes in Camille's patchset. My first guess is that it is related to https://gitlab.com/qemu-project/qemu/-/commit/5f63a67adb58478974b91f5e5c2b1222b5c7f2cc which downgrades docker-in-docker from latest to stable-dind. I'm not sure how the stable tags are treated with docker, but it does seem that those tags haven't been updated in a couple of years. This person on SO seems to have fixed a similar issue in gitlab ci by upgrading: https://stackoverflow.com/q/70707572 The cgroups error leads me to believe that the older dind service is expecting cgroups v1, while the hardware runner is running FCOS with cgroups v2. I ran a test with just that patch omitted on my repo, with better results: https://gitlab.com/eldondev/qemu/-/pipelines/878387857 I will see if the opensbi build which also uses stable-dind and failed in that pipeline is successful when attempting to use latest dind instead. If using stable is compelling, I can work on making the hardware runner support it. Thanks, Eldon
Re: gitlab shared runner time expired
On Thu, May 18, 2023 at 12:26:33PM -0700, Richard Henderson wrote: > So, here we are again, out of runner time with 13 days left in the month. > > Did we come to any resolution since last time? Holding development for that > long just > isn't right, so I'll continue processing the hard way -- testing on private > runners and > local build machines. Hi Richard, We should have capacity for private runners to execute several jobs in parallel. Here [1] is an example of one that ran on a private runner today. I have been thinking about suggesting a strategy to run jobs that lend themselves to amd64 linux runners pinned to private runners that match, so more "shared" minutes can be spent on runners that have different capabilites. If there is another specific arch/OS runner you have in mind, I would be happy to make efforts towards provisioning one in our infrastructure. Particularly it seems like a lot of people are trying to use qemu to support amd64 linux on M1 macs, so it might make sense to have a private mac silicon macOS runner. Also, since we have a hardware runner that can run with kvm capabilites, it might make sense to carve some of those tests out for that tagged runner. Since this is relatively new, I think I was personally in an observational period before seeking input on those goals, but let me know you thoughts. Thanks, Eldon [1] https://gitlab.com/qemu-project/qemu/-/jobs/4310866300
Re: out of CI pipeline minutes again
On Mon, Feb 27, 2023 at 12:43:55PM -0500, Stefan Hajnoczi wrote: > - Personally, I don't think this should exclude other efforts like > Eldon's. We can always add more private runners! Hi! Thanks so much to Alex, Thomas, Gerd, et al for the pointers. Although the month has passed and presumably gitlab credits have replenished, I am interested in continuing my efforts to replicate the shared runner capabilities. After some tinkering I was able to utilise Gerd's stateless runner strategy with a few changes, and had a number of tests pass in a pipeline on my repo: https://gitlab.com/eldondev/qemu/-/pipelines/791573670 Looking at the failures, it seems that some may already be addressed in patchsets, and some may be attributable to things like open file handle count, which would be useful to configure directly on the d-in-d runners, so I will investigate those after integrating the changes from the past couple of days. I have been reading through Alex's patchsets to lower CI time in the hopes that I might be able to contribute something there from my learnings on these pipelines. If there is an intent to switch to the kubernetes gitlab executor, I have worked with kubernetes a number of times in the past, and I can trial that as well. Even with the possibility of turning on Azure and avoiding these monthly crunches, maybe I can provide some help improving the turnaround time of some of the jobs themselves, once I polish off greening the remaining failures on my fork. Forgive me if I knock around a bit here while I figure out how to be useful. Best, Eldon
Re: out of CI pipeline minutes again
On Thu, Feb 23, 2023 at 03:33:00PM +, Daniel P. Berrangé wrote: > IIUC, we already have available compute resources from a couple of > sources we could put into service. The main issue is someone to > actually configure them to act as runners *and* maintain their > operation indefinitely going forward. The sysadmin problem is > what made/makes gitlab's shared runners so incredibly appealing. Hello, I would like to do this, but the path to contribute in this way isn't clear to me at this moment. I made it as far as creating a GitLab fork of QEMU, and then attempting to create a merge request from my branch in order to test the GitLab runner I have provisioned. Not having previously tried to contribute via GitLab, I was a bit stymied that it is not even possibly to create a merge request unless I am a member of the project? I clicked a button to request access. Alex's plan from last month sounds feasible: - provisioning scripts in scripts/ci/setup (if existing not already good enough) - tweak to handle multiple runner instances (or more -j on the build) - changes to .gitlab-ci.d/ so we can use those machines while keeping ability to run on shared runners for those outside the project Daniel, you pointed out the importance of reproducibility, and thus the use of the two-step process, build-docker, and then test-in-docker, so it seems that only docker and the gitlab agent would be strong requirements for running the jobs? I feel like the greatest win for this would be to at least host the cirrus-run jobs on a dedicated runner because the machine seems to simply be burning double minutes until the cirrus job is complete, so I would expect the GitLab runner requirements for those jobs to be low? If there are some other steps that I should take to contribute in this capacity, please let me know. Maybe I could send a patch to tag cirrus jobs in the same way that the s390x jobs are currently tagged, so that we could run those separately? Thanks, Eldon
Re: no more pullreq processing til February
On Thu, Jan 26, 2023 at 01:22:32PM +, Peter Maydell wrote: > Hi; we've run out of gitlab CI pipeline minutes for this month. > This leaves us the choice of: > (a) don't process any more pullreqs til we get more minutes in Feb > (b) merge pullreqs blindly without CI testing > (c) buy more minutes > > For the moment I propose to take option (a). My mail filter will > continue to track pullreqs that get sent to the list, but I won't > do anything with them. > > If anybody has a better suggestion feel free :-) Would it be possible if (d) were to run self-hosted instances of the runner? I am not sure how gitlab pricing works, but I believe on github self-hosted runners are free. I have several baremetal machines colocated that I could dedicate to execute these runs, dual processor xeons with a couple hundred gigs of RAM. I would need approx 48 hours notice to initially provision the machines. I would be happy to provide root credentials and work out IPMI access if that becomes necessary. If this offering isn't suitable, let me know how we can consider adapting something to the project's needs. Thanks, Eldon
Re: Sad to see the advent calendar go
Hi Simon, Thanks for your interest in the calendar! I am not a regular QEMU contributor, but I am a longtime user, and ran the calendar in 2020. I also put out a submission request for the calendar this past year, but there wasn't much response. The cost to host the calendar is trivial, we just need to convert some of the sysadmin responsibilities, and that is in progress. As far as future calendars go, the effort to construct a new calendar with novel, thematic content is substantial. Some of the difficulty lies in the technical element, but a lot of it lies in the synthesis and ideation. In 2020 I had a number of community submissions, and good pointers, but I also put in substantial effort, which I was capable of as a result of a holiday sabbatical. This year was a little different. I'm not sure if others in the community have the same perspective, but that is my perspective, and I would be committed to the 2023 calendar if folks were interested. I assembled some resources for the 2022 calendar, but I think with a little more upfront planning, and some directed request, I can coordinate a much better outcome for 2023. Thanks, Eldon
QEMU Advent Calendar 2022 Call for Images
Hi, We are working to make QEMU Advent Calendar 2022 happen this year, and if you have had an interesting experience with QEMU recently, we would love for you to contribute! QEMU invocations that showcase new functionality, something cool, bring back retro computing memories, or simply entertain with a puzzle or game are welcome. If you have an idea but aren't sure if it fits, email me and we can try to put something together. QEMU Advent Calendar publishes a QEMU disk image each day from December 1-24. Each image is a surprise designed to delight an audience consisting of the QEMU community and beyond. You can see previous years here: https://www.qemu-advent-calendar.org/ You can help us make this year's calendar awesome by: * Sending disk images ( or links to larger images ) * Replying with ideas for disk images (reply off-list to avoid spoilers!) If you have an idea after the start of the advent, go ahead and send it. We may find space to include it, or go ahead and get a jump on 2024! Here is the format we will work with you to create: * A name and a short description of the disk image (e.g. with hints on what to try) * A ./run shell script that prints out the name and description/hints and launches QEMU * A 320x240 screenshot/image/logo for the website Content must be freely redistributable (i.e. no proprietary license that prevents distribution). For GPL based software, you need to provide the source code, too. Check out this disk image as an example of how to distribute an image: https://www.qemu-advent-calendar.org/2018/download/day24.tar.xz PS: QEMU Advent Calendar is a secular calendar (not religious). The idea is to create a fun experience for the QEMU community which can be shared with everyone. You don't need to celebrate Christmas or another religious festival to participate! Thanks, and best wishes! Eldon
Re: Moving QEMU downloads to GitLab Releases?
Hello! I'd be happy to help with this. I'm mostly a consumer of QEMU, but greatly appreciate all the work this community has done, and was able to contribute a little by helping with QEMU advent this past year. I would be happy to help streamline some of this activities if that would be welcome, and would gratefully contribute time and resources. Hosting and serving data like this has been core to my recent experience. I would be happy to suggest and build out a distribution strategy for these packages, and believe I could cut some costs, and even convince a small consultancy I am a part of here that uses QEMU to foot a reasonable bill. A brief introduction, since I haven't had the pleasure of attending FOSDEM or any other QEMU meetups: I am a startup-oriented Cloud Security Architect, based out of Atlanta, previously with companies like DataStax, but now working on AWS video pipelines for a startup here. Thanks, and hopefully I can be of service! Eldon On Thu, Sep 30, 2021 at 03:28:53PM +0100, Stefan Hajnoczi wrote: > On Thu, Sep 30, 2021 at 3:08 PM Stefan Hajnoczi wrote: > > > > Hi Mike, > > QEMU downloads are currently hosted on qemu.org's Apache web server. > > Paolo and I were discussing ways to reduce qemu.org network traffic to > > save money and eventually turn off the qemu.org server since there is no > > full-time sysadmin for it. I'd like to discuss moving QEMU downloads to > > GitLab Releases. > > Daniel Berrange posted this in another discussion: > > "gitlab releases have a per-file size limit that is somewhere on the > order of 10 MB IIRC. Our release tarballs are 100+ MB, so I don't > believe that's going to be viable. > > The gitlab package registry doesn't directly support plain file > downloads afaik, and is also size limited to 50 MB per package > > I'd love to find a good solution for large release artifact hosting, > since I need a better solution for virt-viewer windows MSI releases > wich are similarly large to QEMUs. For that I'm using pagure.io > provided by Fedora, but I don't have confidence in that service > surviving long term." > > So it looks like GitLab Releases won't work for QEMU :(. > > Stefan >
QEMU Advent Calendar 2020 Call for Images
Hi, QEMU Advent Calendar 2020 is around the corner and we are looking for volunteers to contribute disk images that showcase something cool, bring back retro computing memories, or simply entertain with a puzzle or game. QEMU Advent Calendar publishes a QEMU disk image each day from December 1-24. Each image is a surprise designed to delight an audience consisting of the QEMU community and beyond. You can see previous years here: https://www.qemu-advent-calendar.org/ You can help us make this year's calendar awesome by: * Sending disk images ( or links to larger images ) * Replying with ideas for disk images (reply off-list to avoid spoilers!) If you have an idea after the start of the advent, go ahead and send it. We may find space to include it, or go ahead and get a jump on 2021! Here are the requirements for disk images: * Content must be freely redistributable (i.e. no proprietary license that prevents distribution). For GPL based software, you need to provide the source code, too. * Provide a name and a short description of the disk image (e.g. with hints on what to try) * Provide a ./run shell script that prints out the name and description/hints and launches QEMU * Provide a 320x240 screenshot/image/logo for the website * Size should be ideally under 100 MB per disk image (but if some few images are bigger, that should be OK, too) Check out this disk image as an example of how to distribute an image: https://www.qemu-advent-calendar.org/2018/download/day24.tar.xz PS: QEMU Advent Calendar is a secular calendar (not religious). The idea is to create a fun experience for the QEMU community which can be shared with everyone. You don't need to celebrate Christmas or another religious festival to participate! Thanks, and best wishes! Eldon
[PATCH] Add an announcement about the 2020 QEMU advent calendar and a call for images to include.
Signed-off-by: Eldon Stegall --- _posts/2020-11-26-qemu-advent-announce.md | 39 +++ 1 file changed, 39 insertions(+) create mode 100644 _posts/2020-11-26-qemu-advent-announce.md diff --git a/_posts/2020-11-26-qemu-advent-announce.md b/_posts/2020-11-26-qemu-advent-announce.md new file mode 100644 index 000..06c0e45 --- /dev/null +++ b/_posts/2020-11-26-qemu-advent-announce.md @@ -0,0 +1,39 @@ +--- +layout: post +title: "QEMU Advent Calendar 2020 Announcement and Call for Images" +date: 2020-11-22 18:00:00 -0500 +categories: [contributing, community] +--- +QEMU Advent Calendar 2020 is around the corner and we are looking for volunteers to contribute disk images that showcase something cool, bring +back retro computing memories, or simply entertain with a puzzle or game. + +QEMU Advent Calendar publishes a QEMU disk image each day from +December 1-24. Each image is a surprise designed to delight an audience +consisting of the QEMU community and beyond. You can see previous +years at [https://www.qemu-advent-calendar.org/](https://www.qemu-advent-calendar.org/) + +You can help us make this year's calendar awesome by: + * [Sending disk images](mailto:eldon-q...@eldondev.com?Subject=QEMU 2020 advent disk image) + * Sending ideas for disk images + +Here are the requirements for disk images: + * Content must be freely redistributable (i.e. no proprietary + license that prevents distribution). For GPL based software, + you need to provide the source code, too. + * Provide a name and a short description of the disk image + (e.g. with hints on what to try) + * Provide a ./run shell script that prints out the name and + description/hints and launches QEMU + * Provide a 320x240 screenshot/image/logo for the website + * Size should be ideally under 100 MB per disk image + (but if some few images are bigger, that should be OK, too) + +Check out [this disk +image](https://www.qemu-advent-calendar.org/2018/download/day24.tar.xz) as an +example of how to distribute an image. Links to files over 25MB are appreciated +in lieu of email attachments. + +PS: QEMU Advent Calendar is a secular calendar (not +religious). The idea is to create a fun experience for the QEMU +community which can be shared with everyone. You don't need +to celebrate Christmas or another religious festival to participate! -- 2.29.2
