Philippe Mathieu-Daudé <f4...@amsat.org> writes:
> +Stefan/Peter > > On 4/19/21 12:59 PM, Thomas Huth wrote: >> On 19/04/2021 12.51, Daniel P. Berrangé wrote: >>> On Mon, Apr 19, 2021 at 12:48:25PM +0200, Thomas Huth wrote: >>>> On 19/04/2021 12.36, Daniel P. Berrangé wrote: >>>>> On Mon, Apr 19, 2021 at 12:20:55PM +0200, Thomas Huth wrote: >>>>>> On 19/04/2021 12.10, Erik Skultety wrote: >>>>>>> On Mon, Apr 19, 2021 at 10:40:53AM +0100, Daniel P. Berrangé wrote: >>>>>>>> On Mon, Apr 19, 2021 at 01:34:47AM +0200, Philippe Mathieu-Daudé >>>>>>>> wrote: >>>>>>>>> Forks run the same jobs than mainstream, which might be overkill. >>>>>>>>> Allow them to easily rebase their custom set, while keeping using >>>>>>>>> the mainstream templates, and ability to pick specific jobs from >>>>>>>>> the mainstream set. >>>>>>>>> >>>>>>>>> To switch to your set, simply add your .gitlab-ci.yml as >>>>>>>>> .gitlab-ci.d/${CI_PROJECT_NAMESPACE}.yml (where >>>>>>>>> CI_PROJECT_NAMESPACE >>>>>>>>> is your gitlab 'namespace', usually username). This file will be >>>>>>>>> used instead of the default mainstream set. >>>>>>>> >>>>>>>> I find this approach undesirable, because AFAICT, it means you have >>>>>>>> to commit this extra file to any of your downstream branches that >>>>>>>> you want this to be used for. Then you have to be either delete it >>>>>>>> again before sending patches upstream, or tell git-publish to >>>>>>>> exclude the commit that adds this. >>>>>>>> >>>>>>>> IMHO any per-contributor overhead needs to not involve committing >>>>>>>> stuff to their git branches, that isn't intended to go upstream. >>>>>>> >>>>>>> Not just that, ideally, they should also run all the upstream >>>>>>> workloads before >>>>>>> submitting a PR or posting patches because they'd have to respin >>>>>>> because of a >>>>>>> potential failure in upstream pipelines anyway. >>>>>> >>>>>> It's pretty clear that you want to run the full QEMU CI before >>>>>> submitting >>>>>> patches to the QEMU project, but I think we are rather talking >>>>>> about forks >>>>>> here that are meant not meant for immediately contributing to upstream >>>>>> again, like RHEL where we only build the KVM-related targets and >>>>>> certainly >>>>>> do not want to test other things like CPUs that are not capable of >>>>>> KVM, or a >>>>>> branch where Philippe only wants to check his MIPS-related work during >>>>>> development. >>>>>> For contributing patches to upstream, you certainly have to run the >>>>>> full CI, >>>>>> but for other things, it's sometimes really useful to cut down the CI >>>>>> machinery (I'm also doing this in my development branches manually >>>>>> some >>>>>> times to speed up the CI), so I think this series make sense, indeed. >>>>> >>>>> For a downstream like RHEL, I'd just expect them to replace the main >>>>> .gitlab-ci.yml entirely to suit their downstream needs. >>>> >>>> But that still means that we should clean up the main .gitlab-ci.yml >>>> file >>>> anyway, like it is done in this series, to avoid that you always get >>>> conflicts in this big file with your downstream-only modifications. >>>> So at >>>> least up to patch 11 or 12, I think this is a very valuable work that >>>> Philippe is doing here. >>> >>> I don't see a real issue with downstream conflicts. They'll just >>> periodically pick a release to base themselves off and change once >>> every 6 months or more. >> >> It's not only downstream distros that rebase every 6 month. Like >> Philippe, I'm sometimes hacking my .gitlab-ci.yml of my development >> branch to speed up the CI during my development cycles (i.e. I'm >> removing the jobs that I do not need). And I'm regularly rebasing my >> development branchs. Conflicts in .gitlab-ci.yml are then always >> painful, so a leaner main .gitlab-ci.yml file would be helpful for me, >> too, indeed. > > Not sure if following up this thread or start a new one, but I got > blocked again from Gitlab, tagged as a crypto miner for running QEMU > CI... > [1] > https://about.gitlab.com/handbook/support/workflows/investigate_blocked_pipeline.html#trends--high-priority-cases > > I pushed 5 different branches to my repository in less than 1h, > kicking 580 jobs [*]. > > I didn't try to stress Gitlab, it was a simple "one time in the month > rebase unmerged branches, push them before respining on the mailing > list". > > I'm considering changing my workflow: > - not push more than 2 branches per hour (I know 3/h works, so choose > a lower number, as we want to add more tests). > - merge multiple branches locally and push the merged result and > bisect / re-push on failure I stack my branches - so usually I have a: testing/next gdb/next whatever my current hack is Every week I re-base the branches and re-build my current hacking tree. If an actual problem shows up in CI I'll bisect on one of my beefy boxes to fix it and then fix and re-push testing/next and whatever my tip is. > - run less testing > - do not run testing I run a lot of testing locally (or rather on a beefy server) so I'm really only using GitLab for final validation of trees rather than day 2 day. > > This sounds counter productive and doesn't scale to a community of > contributors asked to use Gitlab. > > So far I don't have better idea than this series. > > Who is interested in sending patches to improve our workflow? > > Thanks, > > Phil. > > [*] NB I have 3 extra runners added to my namespace, but it didn't > help, as per [1] I got blocked by reaching an API rate limit. -- Alex Bennée