Re: Information regarding upcoming Gitlab Migration
On Sunday, April 26, 2020 9:40:01 PM EDT Bhushan Shah wrote: > [Please keep sysad...@kde.org list or bs...@kde.org in the CC for > replies] > > Hello Community members, > > In view of upcoming Gitlab migration, we sysadmin team wants to share > the recommended structuring for the repositories on Gitlab. > > We had multiple options, > > - Flat structure: In this option we would have everything under one > single namespace/group: https://invent.kde.org/kde/knetwalk > - Subgroups under top-level group: In this option we would have a groups > under KDE namespace: https://invent.kde.org/kde/games/knetwalk > - Groups at top level: In this option we would establish a series of > groups at the top level, e.g. https://invent.kde.org/games/knetwalk > > We have discussed this with small but representative group of > contributors or maintainers, and based on their suggestions, we > recommend that we go with option 3. Having sub-groups at top level will > allow us to, > > - Provides good visibility on all reviews, tasks and other items within > the groups/modules we define > - Provides improvements to discoverability of projects > - Makes it possible for groups of projects to establish a group level > task board should it fit their needs (for tracking a release for > instance) > - Makes the most semantic sense, as the ‘KDE’ top level group suggested > in option 2 is duplicative considering the Gitlab instance is under > kde.org. > - The discoverability of projects is maximised, as there is no need to > open the top level ‘KDE’ group before going into the subgroup. > > I've worked on draft "move" of the current set of the repositories in > their respective subgroups at the repo-metadata project's branch [1]. > You can browse the directory structure to get idea of how final > structure on Gitlab would look like. > > If we don't have any objections we would like to implement this next > week and move projects to https://invent.kde.org. > > Thanks! > Bhushan for sysadmin team > > [1] > https://cgit.kde.org/sysadmin/repo-metadata.git/tree/projects-invent?h=bshah/invent > ... to answer your original question my opinion is +1 I do appreciate all the work the sysadmins are putting onto this endeavor and I'm glad we are making the move. I hope it happens next week as planned. -Allen
Re: Information regarding upcoming Gitlab Migration
On Sat, May 2, 2020 at 9:04 AM Nate Graham wrote: > > On 5/1/20 2:09 PM, Ben Cooksley wrote: > > Unfortunately sharing of projects/repositories across groups does not > > impact on tasks and reviews. > > This means that merge requests for Planet (which is currently shared > > with "KDE") do not show up in the list of merge requests for "KDE". > > > > Sharing repositories allows for a global listing only. > Are you saying that if we put plasma-framework in Plasma and share it > with the Plasma group, then its MRs won't show up in the Plasma group's > MR list? And that if we put it in the Plasma group and share it with the > Frameworks group, then its MRs won't show up in the Frameworks group's > MR list? That is correct. > > If so, that seems like it defeats one of the points of sharing a > project/repo across groups. :( > > Is this fixed in EE, or is it just a bug affecting all versions? It affects all versions. > > Nate Regards, Ben
Re: Information regarding upcoming Gitlab Migration
On 5/1/20 2:09 PM, Ben Cooksley wrote: Unfortunately sharing of projects/repositories across groups does not impact on tasks and reviews. This means that merge requests for Planet (which is currently shared with "KDE") do not show up in the list of merge requests for "KDE". Sharing repositories allows for a global listing only. Are you saying that if we put plasma-framework in Plasma and share it with the Plasma group, then its MRs won't show up in the Plasma group's MR list? And that if we put it in the Plasma group and share it with the Frameworks group, then its MRs won't show up in the Frameworks group's MR list? If so, that seems like it defeats one of the points of sharing a project/repo across groups. :( Is this fixed in EE, or is it just a bug affecting all versions? Nate
Re: Information regarding upcoming Gitlab Migration
On Sat, May 2, 2020 at 8:30 AM Ingo Klöcker wrote: > > On Freitag, 1. Mai 2020 22:09:52 CEST Ben Cooksley wrote: > > Unfortunately sharing of projects/repositories across groups does not > > impact on tasks and reviews. > > This means that merge requests for Planet (which is currently shared > > with "KDE") do not show up in the list of merge requests for "KDE". > > Which means that everybody who is interested in MRs for projects in different > groups, e.g. PIM frameworks and PIM projects, will have to watch the MR pages > of multiple groups. As I mentioned before, I had to deal with this situation > in the past and it was a major annoyance. While that is unfortunate, the reality is they are no longer 'PIM' projects - they are now Frameworks. No solution is perfect, and it isn't like a massive global list (see https://invent.kde.org/groups/kde/-/merge_requests) is usable by anyone in particular. > > Regards, > Ingo Regards, Ben
Re: Information regarding upcoming Gitlab Migration
On Freitag, 1. Mai 2020 22:09:52 CEST Ben Cooksley wrote: > Unfortunately sharing of projects/repositories across groups does not > impact on tasks and reviews. > This means that merge requests for Planet (which is currently shared > with "KDE") do not show up in the list of merge requests for "KDE". Which means that everybody who is interested in MRs for projects in different groups, e.g. PIM frameworks and PIM projects, will have to watch the MR pages of multiple groups. As I mentioned before, I had to deal with this situation in the past and it was a major annoyance. Regards, Ingo signature.asc Description: This is a digitally signed message part.
Re: Information regarding upcoming Gitlab Migration
On Sat, May 2, 2020 at 6:17 AM Alexander Potashev wrote: > > On Tue, Apr 28, 2020 at 6:47 AM Bhushan Shah wrote: > > Use case 4 : Tom is a student in Germany and is interested in > > contributing to wikitolearn, and he asks where can I find code of the > > wikitolearn? > > Hi, Hi Alexander, > > I have a similar use case. Sometimes I need to share a URL to a > project. For this purpose I used to share e.g. > https://cgit.kde.org/releaseme.git/about > > Does this migration make such permalinks impossible? > > > From what I see, we lose permalinks because > 1. cgit.kde.org will be discontinued We provide the commits.kde.org redirector for permanent links. Anywhere needing a long life link to a particular repository, commit, etc (like documentation) should be using these links and not anything else. > 2. A once valid URL https://invent.kde.org/games/knetwalk may become > unavailable if the project moves to another group, for example > https://invent.kde.org/unmaintained/knetwalk > As mentioned by Nicolas, Gitlab maintains redirects so if such a move were to take place, the above links would continue to work. The only time this is not the case is when a new repository takes it's place. > -- > Alexander Potashev Cheers, Ben
Re: Information regarding upcoming Gitlab Migration
On Sat, May 2, 2020 at 2:33 AM Adriaan de Groot wrote: > > On 2020 mayula d. 1id 07:08:41 CEST Ben Cooksley wrote: > > On Fri, May 1, 2020 at 2:46 AM Nate Graham wrote: > > > If I'm understanding things, we have solutions to most or all of the > > > objections raised so far: > > > > > > - Projects will be allowed to live in--or at least appear in--multiple > > > top-level groups (e.g. plasma-framework could appear in both the > > > Frameworks top-level group and also the Plasma top-level group) > > > > Projects will have the option to appear in multiple groups yes. > > Forgive me for coming full circle on this discussion, but > > - a group can have at most one workboard > - a group offers some facilities for managing issues and reviews that cross > over repositories within that group > - a project (this is one-to-one with "repository", right?) can have as many > workboards as it likes Correct. Projects and repositories are the same thing, it is just a matter of terminology. > > If a project can appear in more than one group, isn't the whole distinction > between flat and namespaced a little .. The ability for a project to appear in other groups is subject to some limitations. See https://invent.kde.org/groups/kde/-/shared for a list of repositories currently shared with "KDE" > > well, how would this proposal fly? > > - Put everything in a single group called "kde" (this matches proposal 2 if I > still remember the original numbering right -- flat, but not at top-level) Proposal 2 had the groupings within a larger "KDE" group, rather than at top level. Proposal 1 was fully flat, just dump everything in one group. > - Other groups hold things from "kde" (this matches proposal 3, giving more > structure / hierarchy) > > People browsing *top* level would see group "kde" (for all I care, bookmark > that one as "I want to browse the list of 1442 repositories") and a bunch of > logical groups based on how the community organizes itself. People working > inside a specific group can do their workboardy-things and focus on the > repositories for that group, while people with an overall interest go to the > KDE group. > Unfortunately sharing of projects/repositories across groups does not impact on tasks and reviews. This means that merge requests for Planet (which is currently shared with "KDE") do not show up in the list of merge requests for "KDE". Sharing repositories allows for a global listing only. > > > Somehow I get the feeling that we started with some technical limitations > which were driving particular choices, where those limitations aren't exactly > what we assumed that they were, and now it looks to me like those limitations > do not have to meaningfully impact *any* of the choices. > > (*if* my understanding is correct; I've been wrong enough times already today) > > [ade] Cheers, Ben
Re: Information regarding upcoming Gitlab Migration
> On 1 May 2020, at 15:17, Alexander Potashev wrote: > > On Tue, Apr 28, 2020 at 6:47 AM Bhushan Shah wrote: >> Use case 4 : Tom is a student in Germany and is interested in >> contributing to wikitolearn, and he asks where can I find code of the >> wikitolearn? > > Hi, > > I have a similar use case. Sometimes I need to share a URL to a > project. For this purpose I used to share e.g. > https://cgit.kde.org/releaseme.git/about > > Does this migration make such permalinks impossible? > > > From what I see, we lose permalinks because > 1. cgit.kde.org will be discontinued > 2. A once valid URL https://invent.kde.org/games/knetwalk may become > unavailable if the project moves to another group, for example > https://invent.kde.org/unmaintained/knetwalk As I understand it, games/knetwalk will become a redirect to unmaintained/knetwalk, so the old link will still work. This is handled by GitLab automatically. Even on gitlab.com, if you rename a repository on your personal account or transfer it to another group or user, the old name will forward to the new one. -- Nicolás
Re: Information regarding upcoming Gitlab Migration
On Tue, Apr 28, 2020 at 6:47 AM Bhushan Shah wrote: > Use case 4 : Tom is a student in Germany and is interested in > contributing to wikitolearn, and he asks where can I find code of the > wikitolearn? Hi, I have a similar use case. Sometimes I need to share a URL to a project. For this purpose I used to share e.g. https://cgit.kde.org/releaseme.git/about Does this migration make such permalinks impossible? >From what I see, we lose permalinks because 1. cgit.kde.org will be discontinued 2. A once valid URL https://invent.kde.org/games/knetwalk may become unavailable if the project moves to another group, for example https://invent.kde.org/unmaintained/knetwalk -- Alexander Potashev
Re: Information regarding upcoming Gitlab Migration: clarifications
Ben Cooksley ha scritto: > On Fri, May 1, 2020 at 4:38 PM Nate Graham wrote: >> >> >> >> On 4/30/20 5:59 PM, Aleix Pol wrote: >>> El jue., 30 de abr. de 2020 a la(s) 18:15, Albert Astals Cid Am I the only person that just has all the repos on the same folder? I thought it was the common thing to do :? >>> >>> I do too >> >> Same here. kdesrc-build's default settings do this, and download all >> repos into ~/kde/src without any of the levels of hierarchy. This would >> seem to require unique repo names, no? > > Not necessarily. > > Git allows you to override the name that the local folder is called > when cloning, so there is no reason why we can't specify something in > the metadata to override the local name that the folder gets called in > your local checkout folder. > This would allow for example: > > - frameworks/kcoreaddons on Gitlab continues to be called > 'kcoreaddons' in your local checkout folder > - maui/dialer on Gitlab gets called 'maui-dialer' in your local checkout > folder > > This allows for the URLs on Gitlab to make sense, while simultaneously > allowing your local source checkout folders to continue to work in the > manner in which they do currently. > Note that we do not expect people to hit this in many cases - this is > about giving us the flexibility for the long term without imposing > unnecessary bureaucracy and limits on projects within the KDE > umbrella. > > Automated tooling such as kdesrc-build and the proposed clone script > (that searches in namespaces) should be able to handle this without > much issue. > In the case of manually done clones, it is reasonable to expect people > to know what they're doing with their clones, and name them > appropriately in the event they have a collision. > > Does this sound acceptable? I'd like to add that this would solve an issue we have on the translation side. Right now, a few translation files use the repository names as the base part of the template file name. For example, the translation files for the desktop and json files, which are called ._desktop_.po(t) and ._json_.po(t) respectively. In the case where the repository name is not unique we would need to record the expected name for the template somewhere. If this information is added to the repository metadata, would have scripty rely on that and don't need to write it down somewhere else. So yes, if you can please add this optional override which sets a unique name to the metadata, that would help, thanks! -- Luigi
Re: Information regarding upcoming Gitlab Migration: clarifications
On 5/1/20 9:02 AM, Johan Ouwerkerk wrote: No, that is not the default. Actually, it is: https://invent.kde.org/kde/kdesrc-build/-/blob/master/kdesrc-build-setup#L389 and download all repos into ~/kde/src without any of the levels of hierarchy. But it is sufficiently common that there is a dedicated setting for it: `ignore-kde-structure`. Kinda does what it says on the tin ;) Yes, and this setting is set by default. :) Nate
Re: Information regarding upcoming Gitlab Migration: clarifications
On Fri, May 1, 2020 at 6:38 AM Nate Graham wrote: > > On 4/30/20 5:59 PM, Aleix Pol wrote: > > El jue., 30 de abr. de 2020 a la(s) 18:15, Albert Astals Cid > >> Am I the only person that just has all the repos on the same folder? I > >> thought it was the common thing to do :? > > > > I do too > > Same here. kdesrc-build's default settings do this, > No, that is not the default. By default the hierarchy is mirrored. > and download all > repos into ~/kde/src without any of the levels of hierarchy. > But it is sufficiently common that there is a dedicated setting for it: `ignore-kde-structure`. Kinda does what it says on the tin ;) > This would > seem to require unique repo names, no? > Yes, it does. Also whether the hierarchy is preserved locally or not, kdesrc-build currently requires that the leaf names ($repo.git) be unique. It cannot fully and consistently distinguish between a maui/dialer and a plasma-mobile/dialer because at certain points in Perl we map to/from module names which are mapped onto those leaf names (i.e. both would be considered "dialer" and it would be anybody's guess at this point what you'd get if you did a `kdesrc-build dialer`). Might be fixable, but is definitely not trivial and would require people to help review and test. Also would require some "cleverness" to preserve the ability to refer to modules by their leaf names when possible (i.e. continuing to be able to do `kdesrc-build kate`), otherwise kdesrc-build becomes a *lot* less user friendly all of a sudden. Regards, - Johan
Re: Is kdeinit still actual?
On Monday, 27 April 2020 03:53:38 PDT Alexander Volkov wrote: > As for KIO slaves, they are started from processes that are already > linked with many KDE libraries and there is no > > much benefit in starting them from kdeinit. That's an incorrect conclusion. Your premise is correct: the process that launches them links to many KDE libraries. But since that launch is a regular fork() + exec(), the premise is irrelevant. The KDE libraries will be loaded again, relocated and initialised, before the ioslave code is run. A quick test here shows kdeinit and one ioslave (file.so, that was already running) are sharing 2380 kB between them. The ioslave has a total of 548 kB of additional private memory that isn't shared with anything else in the system. -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel System Software Products
Re: Information regarding upcoming Gitlab Migration: clarifications
On Fri, May 1, 2020 at 7:18 AM Ben Cooksley wrote: > > On Fri, May 1, 2020 at 4:38 PM Nate Graham wrote: > > > > > > > > On 4/30/20 5:59 PM, Aleix Pol wrote: > > > El jue., 30 de abr. de 2020 a la(s) 18:15, Albert Astals Cid > > >> Am I the only person that just has all the repos on the same folder? I > > >> thought it was the common thing to do :? > > > > > > I do too > > > > Same here. kdesrc-build's default settings do this, and download all > > repos into ~/kde/src without any of the levels of hierarchy. This would > > seem to require unique repo names, no? > > Not necessarily. > > Git allows you to override the name that the local folder is called > when cloning, so there is no reason why we can't specify something in > the metadata to override the local name that the folder gets called in > your local checkout folder. > This would allow for example: > > - frameworks/kcoreaddons on Gitlab continues to be called > 'kcoreaddons' in your local checkout folder > - maui/dialer on Gitlab gets called 'maui-dialer' in your local checkout > folder > > This allows for the URLs on Gitlab to make sense, while simultaneously > allowing your local source checkout folders to continue to work in the > manner in which they do currently. > Note that we do not expect people to hit this in many cases - this is > about giving us the flexibility for the long term without imposing > unnecessary bureaucracy and limits on projects within the KDE > umbrella. > > Automated tooling such as kdesrc-build and the proposed clone script > (that searches in namespaces) should be able to handle this without > much issue. > In the case of manually done clones, it is reasonable to expect people > to know what they're doing with their clones, and name them > appropriately in the event they have a collision. > > Does this sound acceptable? Okay, if that's necessary, we'll have to do it. We'll appreciate simpler bureaucracy. Aleix
Re: Information regarding upcoming Gitlab Migration
On 2020 mayula d. 1id 07:08:41 CEST Ben Cooksley wrote: > On Fri, May 1, 2020 at 2:46 AM Nate Graham wrote: > > If I'm understanding things, we have solutions to most or all of the > > objections raised so far: > > > > - Projects will be allowed to live in--or at least appear in--multiple > > top-level groups (e.g. plasma-framework could appear in both the > > Frameworks top-level group and also the Plasma top-level group) > > Projects will have the option to appear in multiple groups yes. Forgive me for coming full circle on this discussion, but - a group can have at most one workboard - a group offers some facilities for managing issues and reviews that cross over repositories within that group - a project (this is one-to-one with "repository", right?) can have as many workboards as it likes If a project can appear in more than one group, isn't the whole distinction between flat and namespaced a little .. well, how would this proposal fly? - Put everything in a single group called "kde" (this matches proposal 2 if I still remember the original numbering right -- flat, but not at top-level) - Other groups hold things from "kde" (this matches proposal 3, giving more structure / hierarchy) People browsing *top* level would see group "kde" (for all I care, bookmark that one as "I want to browse the list of 1442 repositories") and a bunch of logical groups based on how the community organizes itself. People working inside a specific group can do their workboardy-things and focus on the repositories for that group, while people with an overall interest go to the KDE group. Somehow I get the feeling that we started with some technical limitations which were driving particular choices, where those limitations aren't exactly what we assumed that they were, and now it looks to me like those limitations do not have to meaningfully impact *any* of the choices. (*if* my understanding is correct; I've been wrong enough times already today) [ade] signature.asc Description: This is a digitally signed message part.
Re: Information regarding upcoming Gitlab Migration: clarifications
On Thursday, April 30, 2020 5:15:43 PM EDT Albert Astals Cid wrote: > El dijous, 30 d’abril de 2020, a les 21:31:02 CEST, Ben Cooksley va escriure: > > On Fri, May 1, 2020 at 6:04 AM Ivan Čukić wrote: > > > > > > > We have made a big fuss in the past about having different projects > > > > that do the same thing and now we'll have that but also we'll have > > > > several projects with the same name? > > > > It really feels off to me and I wonder if this is related to the move to > > > > gitlab. > > > > > > +1 to both sentiments - that projects should have different names and that > > > this is a bit off topic for the gitlab migration. > > > > The projects *DO* have very different names. That *HAS NOT* changed. > > To use the example Bhushan gave above, one is called Plasma Mobile > > Dialer and the other one is called Maui Dialer. > > > > With the current git.kde.org setup, we have a flat namespace, so all > > repositories if the name appears to be generic (as dialer is) have to > > be namespaced by prefixing of the repository name. > > > > With Gitlab however we will now namespaces that group repositories, > > making the prefixing unnecessary and as some projects have complained > > about, duplicative. > > > > Otherwise you end up with plasma-mobile/plasma-mobile-dialer as your > > path, which just looks silly. > > Am I the only person that just has all the repos on the same folder? I > thought it was the common thing to do :? > I use kdesrc-build and I see the repos in a hierarchy. In particular, I like frameworks in frameworks in kdepim in kde/pim I don't see that I'm setting any special "layout in a hierarchy" option in my kdesrc-buildrc
Re: Information regarding upcoming Gitlab Migration: clarifications
On 4/30/20 11:17 PM, Ben Cooksley wrote: Not necessarily. Git allows you to override the name that the local folder is called when cloning, so there is no reason why we can't specify something in the metadata to override the local name that the folder gets called in your local checkout folder. This would allow for example: - frameworks/kcoreaddons on Gitlab continues to be called 'kcoreaddons' in your local checkout folder - maui/dialer on Gitlab gets called 'maui-dialer' in your local checkout folder This allows for the URLs on Gitlab to make sense, while simultaneously allowing your local source checkout folders to continue to work in the manner in which they do currently. Note that we do not expect people to hit this in many cases - this is about giving us the flexibility for the long term without imposing unnecessary bureaucracy and limits on projects within the KDE umbrella. Automated tooling such as kdesrc-build and the proposed clone script (that searches in namespaces) should be able to handle this without much issue. In the case of manually done clones, it is reasonable to expect people to know what they're doing with their clones, and name them appropriately in the event they have a collision. Does this sound acceptable? A little weird, but probably acceptable. I suppose it's no worse than having discover build an executable called "plasma-discover", which trips me up like five times a day :) Nate
Re: Information regarding upcoming Gitlab Migration
Thanks for the clarifications, Ben. Then I think the original proposal is perfectly reasonable and I fully support it. Nate