Re: [igt-dev] RFC: Migration to Gitlab
On Fri, Aug 24, 2018 at 8:52 AM, Jani Nikula wrote: > On Wed, 22 Aug 2018, Rodrigo Vivi wrote: >> On Wed, Aug 22, 2018 at 10:19:19AM -0400, Adam Jackson wrote: >>> On Wed, 2018-08-22 at 16:13 +0300, Jani Nikula wrote: >>> >>> > - Sticking to fdo bugzilla and disabling gitlab issues for at least >>> > drm-intel for the time being. Doing that migration in the same go is a >>> > bit much I think. Reassignment across bugzilla and gitlab will be an >>> > issue. >>> >>> Can you elaborate a bit on the issues here? The actual move-the-bugs >>> process has been pretty painless for the parts of xorg we've done so >>> far. >> >> I guess there is nothing against moving the bugs there. The concern is only >> on >> doing everything at once. > > No, it's not just that. > > We have some automation using the bugzilla APIs directly, and > someone(tm) needs to figure out how this should work with gitlab. Maybe > we have a better chance of doing things with gitlab's APIs, maybe we can > reduce our dependence on external logic altogether. > > We have special "i915 platform" and "i915 features" fields in > bugzilla. We use a mailing list default assignee. Some of us use the > "whiteboard" and "keywords" fields. Etc. > > I don't think figuring all this out is rocket science, but someone needs > to actually do it, and get our workflows straightened out *before* we > flip the switch. I'm just trying to figure out if that is a blocker to > migrating the repos. I think the mesa approach of flat-out disabling gitlab issues and merge request is solid. That way there's no confusions with contributions and bug reports stranded in the wrong place. This should allow us to handle all the different feature sets gitlab provides independently and as we see fit: Repo hosting, documentation/artifact hosting, CI integration, patch submissions, issue tracking. We can choose&pick what we think is good, and ignore everything else. And of course if 1-2 years down the road things change, we can reevaluate. I think for a rough timeline if things go well we'll have igt and maintainer-tools migrated by end of year (just the repos, nothing else), and a solid plan for migrating all the drm* repos next year. Everything else is too far away to have solid plans for, or even just a timeline. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [igt-dev] RFC: Migration to Gitlab
On Wed, 22 Aug 2018, Rodrigo Vivi wrote: > On Wed, Aug 22, 2018 at 10:19:19AM -0400, Adam Jackson wrote: >> On Wed, 2018-08-22 at 16:13 +0300, Jani Nikula wrote: >> >> > - Sticking to fdo bugzilla and disabling gitlab issues for at least >> > drm-intel for the time being. Doing that migration in the same go is a >> > bit much I think. Reassignment across bugzilla and gitlab will be an >> > issue. >> >> Can you elaborate a bit on the issues here? The actual move-the-bugs >> process has been pretty painless for the parts of xorg we've done so >> far. > > I guess there is nothing against moving the bugs there. The concern is only on > doing everything at once. No, it's not just that. We have some automation using the bugzilla APIs directly, and someone(tm) needs to figure out how this should work with gitlab. Maybe we have a better chance of doing things with gitlab's APIs, maybe we can reduce our dependence on external logic altogether. We have special "i915 platform" and "i915 features" fields in bugzilla. We use a mailing list default assignee. Some of us use the "whiteboard" and "keywords" fields. Etc. I don't think figuring all this out is rocket science, but someone needs to actually do it, and get our workflows straightened out *before* we flip the switch. I'm just trying to figure out if that is a blocker to migrating the repos. BR, Jani. -- Jani Nikula, Intel Open Source Graphics Center ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Visibility of issues fd.o admins are faced with (Was: Re: RFC: Migration to Gitlab)
[Changing subject/recipients, to avoid hijacking the thread] Hi Dan, On Wed, 22 Aug 2018 at 17:29, Daniel Stone wrote: > > Hi, > > On Wed, 22 Aug 2018 at 16:02, Emil Velikov wrote: > > On 22 August 2018 at 12:44, Daniel Vetter wrote: > > > I think it's time to brainstorm a bit about the gitlab migration. Basic > > > reasons: > > > > > > - fd.o admins want to deprecate shell accounts and hand-rolled > > > infrastructure, because it's a pain to keep secure&updated. > > > > > > - gitlab will allow us to add committers on our own, greatly > > > simplifying that process (and offloading that task from fd.o admins). > > > > Random thought - I really wish the admins spoke early and louder about > > issues. > > From infra to manpower and adhoc tool maintenance. > > I thought I mostly had it covered, but fair enough. What knowledge are > you missing and how should it best be delivered? > > One first-order issue is that as documented at > https://www.freedesktop.org/wiki/AccountRequests/ creating accounts > requires manual admin intervention. You can also search the > 'freedesktop.org' product on Bugzilla to see the amount of time we > spend shuffling around GPG & SSH keys, which is about the worst > possible use of my time. Many other people have had access to drive > the ancient shell-script frontend to LDAP before, but for some reason > they mostly aren't very enthusiastic about doing it all the time. > > In the mesa-dev@ thread about Mesa's migration, which is linked from > my blog post, I went into quite a lot of detail about why Jenkins was > not suitable to roll out across fd.o globally. That knowledge was > gained from trial & error, which was a lot of time burnt. The end > result is that we don't have any CI, except if people hang > Travis/AppVeyor off GitHub mirrors. > > You've personally seen what's involved in setting up Git repository > hooks so we can build docs. We can't give access to let people work on > those, without giving them direct access to the literal Git repository > itself on disk. The hooks mostly involve bespoke sets of rsync jobs > and hand-managed SSH credentials and external services to build docs > and so on and so forth. None of this is accessible to a newcomer who > wants to make a non-code contribution: you have to find someone with > access to the repo to go bash some shell scripts directly and hope you > didn't screw it up. None of this is auditable, so if the repo > mysteriously gets wiped, then hopefully there are some backups > somewhere. But there definitely aren't any logs. Luckily we prevent > most people from having access to most repositories via a mandatory > 'shell' which only allows people to push Git; this was written by hand > by us 12 years ago. > > What else? Our fork of Patchwork was until recently based on an > ancient fork of Django, in a bespoke unreproducible production > environment. Bugzilla is patched for spam and templates (making > upgrades complex), yet we still have a surprising amount of spam pass > through. There's no way to delete spam, but you have to manually move > every bug to the 'spam' group, then go through and delete the user > which involves copying & pasting the email and a few clicks per user. > Mailman is patched to support Recaptcha, bringing more upgrade fun. > ikiwiki breaks all the time because it's designed to have the > public-access web server on the same host as the writeable Git > repositories. > > Our servers are several years old and approaching life expiry, and we > have no more spare disk. You can see in #freedesktop IRC the constant > network connectivity issues people have to PSU almost every day. Our > attempts to root-cause and solve those have got nowhere. > > I could go on, but the 'moved elsewhere' list in > https://gitlab.freedesktop.org/freedesktop/freedesktop/issues/2 > indicates that we do have problems to solve, and that changing > peoples' SSH keys for them isn't the best way for us to be spending > the extremely limited time that we do have. > Starting with the most important - regardless of what may come as nitpicking, I do admire the time, patience and effort that you've been putting in fd.o. Based on your blog-post, there are many issues beyond users usability (think people using cgit/annongit, pw failing to parse email, etc). And yes, I've read it a couple of times as it came out. You mentioned many of those, so let me respin that a bit: - old hacky/adhoc scripts - throw those over a git repo - annoying and/or admin requiring workflow - throw some suggestions about tools in ^^ - ageing servers - increasing maintenance People working on graphics [more or less familiar with some issues] may not be the best admin/tools engineers out there. Hence publicity, be that via blog post/XDC talk/other, is very important IMHO. Don't recall a blog or XDC (lighting) talk over the last 5 years - and seemingly during that time, more issues have been pilling up. I believe that people would have responded to
Re: [igt-dev] RFC: Migration to Gitlab
On Wed, Aug 22, 2018 at 05:37:22PM +0100, Daniel Stone wrote: > Hi Rodrigo, > > On Wed, 22 Aug 2018 at 17:06, Rodrigo Vivi wrote: > > On Wed, Aug 22, 2018 at 10:19:19AM -0400, Adam Jackson wrote: > > > On Wed, 2018-08-22 at 16:13 +0300, Jani Nikula wrote: > > > > - Sticking to fdo bugzilla and disabling gitlab issues for at least > > > > drm-intel for the time being. Doing that migration in the same go is a > > > > bit much I think. Reassignment across bugzilla and gitlab will be an > > > > issue. > > > > > > Can you elaborate a bit on the issues here? The actual move-the-bugs > > > process has been pretty painless for the parts of xorg we've done so > > > far. > > > > I guess there is nothing against moving the bugs there. The concern is only > > on > > doing everything at once. > > > > I'm in favor of moving gits for now and after we are confident that > > everything is there and working we move the bugs. > > As Daniel alluded to, the only issue I really have is moving _all_ the > kernel repos at once. At the end of the year we'll have easy automatic > scaling thanks to the independent services being separated. As it is, > all the GitLab services (apart from CI runners) run on a single > machine, so we have limited options if it becomes overwhelmed with > load. > > Do you have a particular concern about the repos? no concerns from my side about the repos itself. From my committer perspective on libdrm, mesa the migration was really smooth. > e.g. what would you > check for to make sure things are 'there and working'? more in terms of other committers getting used to it, dim working for most of committers, links in documentations and wikis updated... but no concerns with the infra itself. > > > One question about the bugzilla: > > > > Will all the referrences on all commit messages get outdated after > > bugzilla is dead? > > Or bugzilla will stay up for referrence but closed for interaction? > > or all old closed stuff are always moved and bugzilla.fd.o as well and > > bugzilla.fd.o will be mirroring gitlab? > > When bugs are migrated from Bugzilla to GitLab, only open bugs are > migrated. Closed ones are left in place, as is; open ones have a > comment at the end saying that the bug has moved to GitLab, a URL > linking to the new GitLab issue, and telling them to please chase it > up there. > > Even when we move everyone completely off Bugzilla, we will keep it as > a read-only mirror forever. Even with Phabricator, which very few > people ever used, has had all its bugs and code review captured and > archived, so we can continue to preserve all the old content and > links, without having to run the actual service. Great! Thanks for all clarification, Rodrigo. > > Cheers, > Daniel ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [igt-dev] RFC: Migration to Gitlab
Hi Rodrigo, On Wed, 22 Aug 2018 at 17:06, Rodrigo Vivi wrote: > On Wed, Aug 22, 2018 at 10:19:19AM -0400, Adam Jackson wrote: > > On Wed, 2018-08-22 at 16:13 +0300, Jani Nikula wrote: > > > - Sticking to fdo bugzilla and disabling gitlab issues for at least > > > drm-intel for the time being. Doing that migration in the same go is a > > > bit much I think. Reassignment across bugzilla and gitlab will be an > > > issue. > > > > Can you elaborate a bit on the issues here? The actual move-the-bugs > > process has been pretty painless for the parts of xorg we've done so > > far. > > I guess there is nothing against moving the bugs there. The concern is only on > doing everything at once. > > I'm in favor of moving gits for now and after we are confident that > everything is there and working we move the bugs. As Daniel alluded to, the only issue I really have is moving _all_ the kernel repos at once. At the end of the year we'll have easy automatic scaling thanks to the independent services being separated. As it is, all the GitLab services (apart from CI runners) run on a single machine, so we have limited options if it becomes overwhelmed with load. Do you have a particular concern about the repos? e.g. what would you check for to make sure things are 'there and working'? > One question about the bugzilla: > > Will all the referrences on all commit messages get outdated after > bugzilla is dead? > Or bugzilla will stay up for referrence but closed for interaction? > or all old closed stuff are always moved and bugzilla.fd.o as well and > bugzilla.fd.o will be mirroring gitlab? When bugs are migrated from Bugzilla to GitLab, only open bugs are migrated. Closed ones are left in place, as is; open ones have a comment at the end saying that the bug has moved to GitLab, a URL linking to the new GitLab issue, and telling them to please chase it up there. Even when we move everyone completely off Bugzilla, we will keep it as a read-only mirror forever. Even with Phabricator, which very few people ever used, has had all its bugs and code review captured and archived, so we can continue to preserve all the old content and links, without having to run the actual service. Cheers, Daniel ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [igt-dev] RFC: Migration to Gitlab
Hi, On Wed, 22 Aug 2018 at 15:44, Daniel Vetter wrote: > On Wed, Aug 22, 2018 at 3:13 PM, Jani Nikula > wrote: > > Just a couple of concerns from drm/i915 perspective for starters: > > > > - Patchwork integration. I think we'll want to keep patchwork for at > > least intel-gfx etc. for the time being. IIUC the one thing we need is > > some server side hook to update patchwork on git push. > > > > - Sticking to fdo bugzilla and disabling gitlab issues for at least > > drm-intel for the time being. Doing that migration in the same go is a > > bit much I think. Reassignment across bugzilla and gitlab will be an > > issue. > > Good points, forgot about both. Patchwork reading the mailing list > should keep working as-is, but the update hook needs looking into. All the hooks are retained. gitlab.fd.o pushes to git.fd.o, and git.fd.o still executes all the old hooks. This includes Patchwork, readthedocs, AppVeyor, and whatever else. > For merge requests I think best approach is to enable them very > selectively at first for testing out, and then making a per-subproject > decision whether they make sense. E.g. I think for maintainer-tools > integrating make check and the doc building into gitlab CI would be > sweet, and worth looking into gitlab merge requests just to automate > that. Again best left out of scope for the initial migration. You don't need MRs to do that. You can build a .gitlab-ci.yml file which will run make check or build the docs or whatever, and have that run on pushes. Anyone can then fork the repo, push their changes to that fork, and see the CI results from there. It's like Travis: the CI configuration is a (mutable) part of the codebase, not a separate 'thing' hanging off a specific repo. So if you write the CI pipeline first, you can have people running CI on push completely independently of switching the workflow to use MRs. Cheers, Daniel ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: RFC: Migration to Gitlab
Hi, On Wed, 22 Aug 2018 at 16:02, Emil Velikov wrote: > On 22 August 2018 at 12:44, Daniel Vetter wrote: > > I think it's time to brainstorm a bit about the gitlab migration. Basic > > reasons: > > > > - fd.o admins want to deprecate shell accounts and hand-rolled > > infrastructure, because it's a pain to keep secure&updated. > > > > - gitlab will allow us to add committers on our own, greatly > > simplifying that process (and offloading that task from fd.o admins). > > Random thought - I really wish the admins spoke early and louder about issues. > From infra to manpower and adhoc tool maintenance. I thought I mostly had it covered, but fair enough. What knowledge are you missing and how should it best be delivered? One first-order issue is that as documented at https://www.freedesktop.org/wiki/AccountRequests/ creating accounts requires manual admin intervention. You can also search the 'freedesktop.org' product on Bugzilla to see the amount of time we spend shuffling around GPG & SSH keys, which is about the worst possible use of my time. Many other people have had access to drive the ancient shell-script frontend to LDAP before, but for some reason they mostly aren't very enthusiastic about doing it all the time. In the mesa-dev@ thread about Mesa's migration, which is linked from my blog post, I went into quite a lot of detail about why Jenkins was not suitable to roll out across fd.o globally. That knowledge was gained from trial & error, which was a lot of time burnt. The end result is that we don't have any CI, except if people hang Travis/AppVeyor off GitHub mirrors. You've personally seen what's involved in setting up Git repository hooks so we can build docs. We can't give access to let people work on those, without giving them direct access to the literal Git repository itself on disk. The hooks mostly involve bespoke sets of rsync jobs and hand-managed SSH credentials and external services to build docs and so on and so forth. None of this is accessible to a newcomer who wants to make a non-code contribution: you have to find someone with access to the repo to go bash some shell scripts directly and hope you didn't screw it up. None of this is auditable, so if the repo mysteriously gets wiped, then hopefully there are some backups somewhere. But there definitely aren't any logs. Luckily we prevent most people from having access to most repositories via a mandatory 'shell' which only allows people to push Git; this was written by hand by us 12 years ago. What else? Our fork of Patchwork was until recently based on an ancient fork of Django, in a bespoke unreproducible production environment. Bugzilla is patched for spam and templates (making upgrades complex), yet we still have a surprising amount of spam pass through. There's no way to delete spam, but you have to manually move every bug to the 'spam' group, then go through and delete the user which involves copying & pasting the email and a few clicks per user. Mailman is patched to support Recaptcha, bringing more upgrade fun. ikiwiki breaks all the time because it's designed to have the public-access web server on the same host as the writeable Git repositories. Our servers are several years old and approaching life expiry, and we have no more spare disk. You can see in #freedesktop IRC the constant network connectivity issues people have to PSU almost every day. Our attempts to root-cause and solve those have got nowhere. I could go on, but the 'moved elsewhere' list in https://gitlab.freedesktop.org/freedesktop/freedesktop/issues/2 indicates that we do have problems to solve, and that changing peoples' SSH keys for them isn't the best way for us to be spending the extremely limited time that we do have. > > For the full in-depth writeup of everything, see > > > > https://www.fooishbar.org/blog/gitlab-fdo-introduction/ If you haven't seen this, or the post it was linked from, they would be a good start: https://lists.freedesktop.org/archives/freedesktop/2018-July/000370.html There's also the long thread on mesa-dev a long time back which covers a lot of this ground already. > > - Figuring out the actual migration - we've been adding a pile of > > committers since fd.o LDAP was converted to gitlab once back in > > spring. We need to at least figure out how to move the new > > accounts/committers. > > As a observer, allow me to put some ideas. You've mostly covered them > all, my emphasis is to seriously stick with _one_ thing at a time. > Attempting to do multiple things in parallel will end up with > sub-optimal results. > > - (at random point) cleanup the committers list - people who have not > contributed in the last 1 year? libdrm was previously covered under the Mesa ACL. Here are the other committer lists, which you can see via 'getent group' on annarchy or another machine: amdkfd:x:922:fxkuehl,agd5f,deathsimple,danvet,jazhou,jbridgman,hwentland,tstdenis,gitlab-mirror,rui drm-meson:x:936:narmstrong drm:x:940:airlie
Re: [igt-dev] RFC: Migration to Gitlab
On Wed, Aug 22, 2018 at 10:19:19AM -0400, Adam Jackson wrote: > On Wed, 2018-08-22 at 16:13 +0300, Jani Nikula wrote: > > > - Sticking to fdo bugzilla and disabling gitlab issues for at least > > drm-intel for the time being. Doing that migration in the same go is a > > bit much I think. Reassignment across bugzilla and gitlab will be an > > issue. > > Can you elaborate a bit on the issues here? The actual move-the-bugs > process has been pretty painless for the parts of xorg we've done so > far. I guess there is nothing against moving the bugs there. The concern is only on doing everything at once. I'm in favor of moving gits for now and after we are confident that everything is there and working we move the bugs. One question about the bugzilla: Will all the referrences on all commit messages get outdated after bugzilla is dead? Or bugzilla will stay up for referrence but closed for interaction? or all old closed stuff are always moved and bugzilla.fd.o as well and bugzilla.fd.o will be mirroring gitlab? Thanks, Rodrigo. > > - ajax > ___ > dim-tools mailing list > dim-to...@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dim-tools ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: RFC: Migration to Gitlab
Hi Dan, On 22 August 2018 at 12:44, Daniel Vetter wrote: > Hi all, > > I think it's time to brainstorm a bit about the gitlab migration. Basic > reasons: > > - fd.o admins want to deprecate shell accounts and hand-rolled > infrastructure, because it's a pain to keep secure&updated. > > - gitlab will allow us to add committers on our own, greatly > simplifying that process (and offloading that task from fd.o admins). > Random thought - I really wish the admins spoke early and louder about issues. From infra to manpower and adhoc tool maintenance. > There's also some more benefits we might want to reap, like better CI > integration for basic build testing - no more "oops didn't build > drm-misc defconfigs" or "sry, forgot make check in maintainer-tools". > But that's all fully optional. > > For the full in-depth writeup of everything, see > > https://www.fooishbar.org/blog/gitlab-fdo-introduction/ > > I think now is also a good time, with mesa, xorg, wayland/weston and > others moved, to start thinking about how we'll move drm. There's a > few things to figure out though: > > - We probably want to split out maintainer-tools. That would address > the concern that there's 50+ committers to an auto-updating shell > script ... > > - We need to figure out how to handle the ACL trickery around drm-tip in > gitlab. > > - Probably good to stage the migration, with maintainer-tools, igt > leading. That will also make fd.o admins happy, who want to rework > their cloud infrastructure a bit before migrating the big kernel repos > over. > > - Figuring out the actual migration - we've been adding a pile of > committers since fd.o LDAP was converted to gitlab once back in > spring. We need to at least figure out how to move the new > accounts/committers. > As a observer, allow me to put some ideas. You've mostly covered them all, my emphasis is to seriously stick with _one_ thing at a time. Attempting to do multiple things in parallel will end up with sub-optimal results. - (at random point) cleanup the committers list - people who have not contributed in the last 1 year? - setup drm group, copy/migrate accounts - one could even reuse the existing credentials - move git repos to gitlab, the push URL change, cgit mirror preserves the normal fetch ones as well as PW hooks - work out how new accounts are handled - still in bugzilla, otherwise At this stage only workflow change is a) once-off account setup and b) pushURL update As a follow-up one can setup anything fancy. - migrate PW/other hooks - migrate bugs - if applicable - add new hooks - DRM docs, other - etc > - Similar, maintainer-tools needs to move. We probably want to move > all the dim maintained kernel repos in one go, to avoid headaches with > double-accounts needed for committers. > One should be able to create a separate repo for these. And then either: - one by one add the required features into the gitlab MR machinery, - or, wire the execution in pre/post merge stage. IIRC there are some upstream requests about the former. > - CI, linux-next and everyone else should be fine, since the > cgit/non-ssh paths will keep working (they'll be read-only mirrors). > Need to double-check that with everyone. > > - Some organization structure would be good. > > https://cgit.freedesktop.org/drm > > libdrm won't be part of the gitlab drm group because that's already > moved under mesa (and you can't symlink/mulit-home anymore on gitlab): > > https://gitlab.freedesktop.org/mesa/drm > > But there's also drm_hwcomposer, which we might want to migrate into > drm too - gitlab requires a containing group, and > drm_hwcomposer/drm_hwcomposer is a bit silly. > It did strike me a lot when drm_hwcomposer/drm_hwcomposer was introduced. Fortunately moving repos in gitlab is reasonably pain-free HTH Emil ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [igt-dev] RFC: Migration to Gitlab
On Wed, Aug 22, 2018 at 3:13 PM, Jani Nikula wrote: > On Wed, 22 Aug 2018, Daniel Vetter wrote: >> Hi all, >> >> I think it's time to brainstorm a bit about the gitlab migration. Basic >> reasons: >> >> - fd.o admins want to deprecate shell accounts and hand-rolled >> infrastructure, because it's a pain to keep secure&updated. >> >> - gitlab will allow us to add committers on our own, greatly >> simplifying that process (and offloading that task from fd.o admins). >> >> There's also some more benefits we might want to reap, like better CI >> integration for basic build testing - no more "oops didn't build >> drm-misc defconfigs" or "sry, forgot make check in maintainer-tools". >> But that's all fully optional. >> >> For the full in-depth writeup of everything, see >> >> https://www.fooishbar.org/blog/gitlab-fdo-introduction/ >> >> I think now is also a good time, with mesa, xorg, wayland/weston and >> others moved, to start thinking about how we'll move drm. There's a >> few things to figure out though: >> >> - We probably want to split out maintainer-tools. That would address >> the concern that there's 50+ committers to an auto-updating shell >> script ... >> >> - We need to figure out how to handle the ACL trickery around drm-tip in >> gitlab. >> >> - Probably good to stage the migration, with maintainer-tools, igt >> leading. That will also make fd.o admins happy, who want to rework >> their cloud infrastructure a bit before migrating the big kernel repos >> over. >> >> - Figuring out the actual migration - we've been adding a pile of >> committers since fd.o LDAP was converted to gitlab once back in >> spring. We need to at least figure out how to move the new >> accounts/committers. >> >> - Similar, maintainer-tools needs to move. We probably want to move >> all the dim maintained kernel repos in one go, to avoid headaches with >> double-accounts needed for committers. >> >> - CI, linux-next and everyone else should be fine, since the >> cgit/non-ssh paths will keep working (they'll be read-only mirrors). >> Need to double-check that with everyone. >> >> - Some organization structure would be good. >> >> https://cgit.freedesktop.org/drm >> >> libdrm won't be part of the gitlab drm group because that's already >> moved under mesa (and you can't symlink/mulit-home anymore on gitlab): >> >> https://gitlab.freedesktop.org/mesa/drm >> >> But there's also drm_hwcomposer, which we might want to migrate into >> drm too - gitlab requires a containing group, and >> drm_hwcomposer/drm_hwcomposer is a bit silly. >> >> Note: Access rights can be done at any level in the hierarchy, the >> organization is orthogonal to commit rights. >> >> - Anything else I've forgotten. >> >> A lot of this still needs to be figured out first. As a first step I'm >> looking for volunteers who want to join the fun, besides comments and >> thoughts on the overall topic of course. > > Just a couple of concerns from drm/i915 perspective for starters: > > - Patchwork integration. I think we'll want to keep patchwork for at > least intel-gfx etc. for the time being. IIUC the one thing we need is > some server side hook to update patchwork on git push. > > - Sticking to fdo bugzilla and disabling gitlab issues for at least > drm-intel for the time being. Doing that migration in the same go is a > bit much I think. Reassignment across bugzilla and gitlab will be an > issue. Good points, forgot about both. Patchwork reading the mailing list should keep working as-is, but the update hook needs looking into. Disabling gitlab issues is a non-brainer, same with merge requests. Mesa is already doing that. For bugs I think it's best to entirely leave them out for now, and maybe reconsider when/if mesa has moved. Before that I don't think gitlab issues make any sense at all. For merge requests I think best approach is to enable them very selectively at first for testing out, and then making a per-subproject decision whether they make sense. E.g. I think for maintainer-tools integrating make check and the doc building into gitlab CI would be sweet, and worth looking into gitlab merge requests just to automate that. Again best left out of scope for the initial migration. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [igt-dev] RFC: Migration to Gitlab
On Wed, 2018-08-22 at 16:13 +0300, Jani Nikula wrote: > - Sticking to fdo bugzilla and disabling gitlab issues for at least > drm-intel for the time being. Doing that migration in the same go is a > bit much I think. Reassignment across bugzilla and gitlab will be an > issue. Can you elaborate a bit on the issues here? The actual move-the-bugs process has been pretty painless for the parts of xorg we've done so far. - ajax ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [igt-dev] RFC: Migration to Gitlab
On Wed, 22 Aug 2018, Daniel Vetter wrote: > Hi all, > > I think it's time to brainstorm a bit about the gitlab migration. Basic > reasons: > > - fd.o admins want to deprecate shell accounts and hand-rolled > infrastructure, because it's a pain to keep secure&updated. > > - gitlab will allow us to add committers on our own, greatly > simplifying that process (and offloading that task from fd.o admins). > > There's also some more benefits we might want to reap, like better CI > integration for basic build testing - no more "oops didn't build > drm-misc defconfigs" or "sry, forgot make check in maintainer-tools". > But that's all fully optional. > > For the full in-depth writeup of everything, see > > https://www.fooishbar.org/blog/gitlab-fdo-introduction/ > > I think now is also a good time, with mesa, xorg, wayland/weston and > others moved, to start thinking about how we'll move drm. There's a > few things to figure out though: > > - We probably want to split out maintainer-tools. That would address > the concern that there's 50+ committers to an auto-updating shell > script ... > > - We need to figure out how to handle the ACL trickery around drm-tip in > gitlab. > > - Probably good to stage the migration, with maintainer-tools, igt > leading. That will also make fd.o admins happy, who want to rework > their cloud infrastructure a bit before migrating the big kernel repos > over. > > - Figuring out the actual migration - we've been adding a pile of > committers since fd.o LDAP was converted to gitlab once back in > spring. We need to at least figure out how to move the new > accounts/committers. > > - Similar, maintainer-tools needs to move. We probably want to move > all the dim maintained kernel repos in one go, to avoid headaches with > double-accounts needed for committers. > > - CI, linux-next and everyone else should be fine, since the > cgit/non-ssh paths will keep working (they'll be read-only mirrors). > Need to double-check that with everyone. > > - Some organization structure would be good. > > https://cgit.freedesktop.org/drm > > libdrm won't be part of the gitlab drm group because that's already > moved under mesa (and you can't symlink/mulit-home anymore on gitlab): > > https://gitlab.freedesktop.org/mesa/drm > > But there's also drm_hwcomposer, which we might want to migrate into > drm too - gitlab requires a containing group, and > drm_hwcomposer/drm_hwcomposer is a bit silly. > > Note: Access rights can be done at any level in the hierarchy, the > organization is orthogonal to commit rights. > > - Anything else I've forgotten. > > A lot of this still needs to be figured out first. As a first step I'm > looking for volunteers who want to join the fun, besides comments and > thoughts on the overall topic of course. Just a couple of concerns from drm/i915 perspective for starters: - Patchwork integration. I think we'll want to keep patchwork for at least intel-gfx etc. for the time being. IIUC the one thing we need is some server side hook to update patchwork on git push. - Sticking to fdo bugzilla and disabling gitlab issues for at least drm-intel for the time being. Doing that migration in the same go is a bit much I think. Reassignment across bugzilla and gitlab will be an issue. BR, Jani. -- Jani Nikula, Intel Open Source Graphics Center ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [Intel-gfx] RFC: Migration to Gitlab
On Wed, Aug 22, 2018 at 01:44:56PM +0200, Daniel Vetter wrote: > Hi all, > > I think it's time to brainstorm a bit about the gitlab migration. Basic > reasons: > > - fd.o admins want to deprecate shell accounts and hand-rolled > infrastructure, because it's a pain to keep secure&updated. > > - gitlab will allow us to add committers on our own, greatly > simplifying that process (and offloading that task from fd.o admins). > > There's also some more benefits we might want to reap, like better CI > integration for basic build testing - no more "oops didn't build > drm-misc defconfigs" or "sry, forgot make check in maintainer-tools". > But that's all fully optional. > > For the full in-depth writeup of everything, see > > https://www.fooishbar.org/blog/gitlab-fdo-introduction/ > > I think now is also a good time, with mesa, xorg, wayland/weston and > others moved, to start thinking about how we'll move drm. There's a > few things to figure out though: > > - We probably want to split out maintainer-tools. That would address > the concern that there's 50+ committers to an auto-updating shell > script ... > /me laughs nervously > - We need to figure out how to handle the ACL trickery around drm-tip in > gitlab. > > - Probably good to stage the migration, with maintainer-tools, igt > leading. That will also make fd.o admins happy, who want to rework > their cloud infrastructure a bit before migrating the big kernel repos > over. > > - Figuring out the actual migration - we've been adding a pile of > committers since fd.o LDAP was converted to gitlab once back in > spring. We need to at least figure out how to move the new > accounts/committers. > > - Similar, maintainer-tools needs to move. We probably want to move > all the dim maintained kernel repos in one go, to avoid headaches with > double-accounts needed for committers. > > - CI, linux-next and everyone else should be fine, since the > cgit/non-ssh paths will keep working (they'll be read-only mirrors). > Need to double-check that with everyone. They can also pull the trees from git://gitlab.fd.o/blah as normal, just need to give them new pointers once we're stable. > > - Some organization structure would be good. > > https://cgit.freedesktop.org/drm > > libdrm won't be part of the gitlab drm group because that's already > moved under mesa (and you can't symlink/mulit-home anymore on gitlab): > > https://gitlab.freedesktop.org/mesa/drm > > But there's also drm_hwcomposer, which we might want to migrate into > drm too - gitlab requires a containing group, and > drm_hwcomposer/drm_hwcomposer is a bit silly. This seems fine to me. Our expansion plans likely aren't big enough to warrant a separate group. > > Note: Access rights can be done at any level in the hierarchy, the > organization is orthogonal to commit rights. > > - Anything else I've forgotten. > > A lot of this still needs to be figured out first. As a first step I'm > looking for volunteers who want to join the fun, besides comments and > thoughts on the overall topic of course. I'm pretty keen on getting this done, so I'll volunteer some cycles if there's something that needs doing. Sean > > Cheers, Daniel > -- > Daniel Vetter > Software Engineer, Intel Corporation > +41 (0) 79 365 57 48 - http://blog.ffwll.ch > ___ > Intel-gfx mailing list > intel-...@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/intel-gfx -- Sean Paul, Software Engineer, Google / Chromium OS ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
RFC: Migration to Gitlab
Hi all, I think it's time to brainstorm a bit about the gitlab migration. Basic reasons: - fd.o admins want to deprecate shell accounts and hand-rolled infrastructure, because it's a pain to keep secure&updated. - gitlab will allow us to add committers on our own, greatly simplifying that process (and offloading that task from fd.o admins). There's also some more benefits we might want to reap, like better CI integration for basic build testing - no more "oops didn't build drm-misc defconfigs" or "sry, forgot make check in maintainer-tools". But that's all fully optional. For the full in-depth writeup of everything, see https://www.fooishbar.org/blog/gitlab-fdo-introduction/ I think now is also a good time, with mesa, xorg, wayland/weston and others moved, to start thinking about how we'll move drm. There's a few things to figure out though: - We probably want to split out maintainer-tools. That would address the concern that there's 50+ committers to an auto-updating shell script ... - We need to figure out how to handle the ACL trickery around drm-tip in gitlab. - Probably good to stage the migration, with maintainer-tools, igt leading. That will also make fd.o admins happy, who want to rework their cloud infrastructure a bit before migrating the big kernel repos over. - Figuring out the actual migration - we've been adding a pile of committers since fd.o LDAP was converted to gitlab once back in spring. We need to at least figure out how to move the new accounts/committers. - Similar, maintainer-tools needs to move. We probably want to move all the dim maintained kernel repos in one go, to avoid headaches with double-accounts needed for committers. - CI, linux-next and everyone else should be fine, since the cgit/non-ssh paths will keep working (they'll be read-only mirrors). Need to double-check that with everyone. - Some organization structure would be good. https://cgit.freedesktop.org/drm libdrm won't be part of the gitlab drm group because that's already moved under mesa (and you can't symlink/mulit-home anymore on gitlab): https://gitlab.freedesktop.org/mesa/drm But there's also drm_hwcomposer, which we might want to migrate into drm too - gitlab requires a containing group, and drm_hwcomposer/drm_hwcomposer is a bit silly. Note: Access rights can be done at any level in the hierarchy, the organization is orthogonal to commit rights. - Anything else I've forgotten. A lot of this still needs to be figured out first. As a first step I'm looking for volunteers who want to join the fun, besides comments and thoughts on the overall topic of course. Cheers, Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
