Daniel Stone migrated freedesktop.org's stuff to a self-hosted GitLab. Below are Daniel's thoughts.
-------- Forwarded Message -------- From: Daniel Stone To: Sajith Sasidharan Cc: Ben Gamari Subject: Re: About Trac to GitLab migration Date: Thu, 23 Jul 2020 12:34:08 +0100 Hi all, On Thu, 23 Jul 2020 at 03:53, Sajith Sasidharan <[email protected]> wrote: > On Wed, 2020-07-22 at 19:26 -0400, Ben Gamari wrote: > > Anyways, I hope this helps! I would be happy to chat if you have any > > specific questions. > > Thank you very much! What you wrote is very helpful. The GHC blog post did > not turn up in my search results. Glad that I asked: I got more this way. > :) Nice to meet you Sajith! Unfortunately I don't have anything intelligent to say about Trac, but I can definitely help with non-Trac migration and hosting things. > We have been talking to GitLab about their open source program. I don't > believe we can (or should!) spare the resources for self-hosting GitLab. > > It sounds like we won't be able to preserve issue numbers if we're to move > straight to gitlab.com, but I don't know how important of an issue this > would > be for us. Tahoe-LAFS wiki is rather modest in comparison to GHC wiki, so > the > manual intervention also must be modest. GitLab issue numbers are per-project rather than global, so if you're splitting Trac issues between different projects, then I imagine you'd lose the 1:1 correspondence. Rather than using GitLab's native import, we reused and adapted GNOME's bztogl to migrate from Bugzilla: https://gitlab.gnome.org/World/bztogl + https://gitlab.freedesktop.org/freedesktop/bztogl - maybe that's of some help to you, maybe not. One difficult issue is preserving users. Since we run our own instance, we could make sure that everyone had the right local username, and that user accounts were preserved in the migration. If you're hosting on gitlab.com, that option is not open to you, so that's one thing to think about. > I actually have some more questions: > > Have you been able to update your patched GitLab installation to a > "standard" > GitHub release later? freedesktop.org runs an unpatched GitLab codebase, though from time-to-time we do monkeypatch some important hotfixes in. We just use the Helm charts to run our instance in Kubernetes, so we can deploy whatever we can get inside a Docker container. We haven't had any issues with stability of database/on-disk formats to speak of. > What is the setup like for macOS and Windows CI runners? For macOS I have no idea, since getting hosted macOS instances is amazingly painful - thankfully we don't really have to care about it. For Windows, we do run a combination of native (execute in PowerShell on the host, with all the downsides Ben mentioned) and also Docker-on-Windows runners. We only use the native executor to build Docker containers (since Windows can't do Docker-in-Docker, and better tools like buildah/kaniko/etc don't run there), and everything else runs in the Docker executor, cf: https://gitlab.freedesktop.org/mesa/mesa/-/blob/master/.gitlab-ci.yml#L366 (core job definitions throughout this file) https://gitlab.freedesktop.org/mesa/mesa/-/tree/master/.gitlab-ci/windows (container-build job runs mesa_container.ps1 -> Dockerfile -> mesa_deps.ps1; in-container builds just run mesa_build.ps1) Happy to speak more about that if it helps, though I should note that high on my list is stealing the ghc script to clean up after Windows docker instances. ;) We run those on Windows Server Datacentre edition instances; if you're not using DC, then you're restricted to Hyper-V containers rather than process-isolation, so it's a lot slower. But DC is also quite expensive, so ... > I am guessing that you must have considered making a Pandoc reader for Trac > wiki markup before writing a custom tool -- is that a viable approach? We just ignored the GitLab wikis because they are frankly bad. The history on them is really poor, and I also don't like that you have to have access to the project to be able to do anything with the wiki. Instead of using the wikis, we encourage all our projects to keep their documentation in GitLab Pages projects: that way people can fork the repository and send MRs against the documentation, so docs can be reviewed, approved, and shipped the same way as code. That requires using a static site generator like Jekyll, Hugo, Sphinx, or whatever. There are a lot of templates around there for using pretty much every SSG you can think of in CI with Pages. My impression of the GitLab wiki is that they wrote it very quickly as a proof of concept to match GitHub feature-for-feature, and at that point they were abandoned. The discoverability, searchability, and usability of them is really awful. > May I share our communication with tahoe-dev mailing list? It is a low > traffic public list (currently a little broken too, perhaps), with a public > list archive. Happy for anything I say to be made public. I can't really think of any kind of 'gotchas' or anything you should particularly know. Apart from that you should definitely get familiar with both the CI pipeline and the GitLab API. Both of them are really powerful, and we've now found that almost our entire workflows are scripted and automated through a combination of those two. Cheers, Daniel _______________________________________________ tahoe-dev mailing list [email protected] https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
