Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
As of 2 days ago: SMTP is setup, and sign-up is re-enabled with email confirmation. Git repos are still not migrated over. I'm starting another thread for updates on this. - Devan Florian Dold transcribed 2.5K bytes: > On 4/7/19 6:29 AM, Devan C. - dvn wrote: > > Until email/smtp is setup we will not have confirmation emails. > > Ah. I assumed the setup was mostly complete and you wanted feedback. > > > Neither of these are actual problems right now, because it is easy > > enough to manually administer, prune, and moderate. I've deleted your > > "fake" Christian Grothoff account, and all the repos along with it. > > Took only a moment and a couple of clicks. > > You wouldn't be able to distinguish these fake accounts from real > account, that's the point. > > For company-internal intranet services it would be okay, but having a > service exposed on the public internet where everybody can create > accounts for any email address without owning it is totally unacceptable. > > > I've also tightened up the permissions on the GNUnet group. "No harm, no > > foul" as they say... > > > > To prevent any additional alarm, I have disabled registration. We can > > re-enable once we have email confirmation setup. > > Good. > > - Florian > signature.asc Description: PGP signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
Florian Dold transcribed 2.3K bytes: > On 4/10/19 1:10 AM, Raphael Arias wrote: > > I don't see why the database of a read-only legacy system would need to > > be regularly backed up. If it remains online purely as an archive > > (possibly with a note to go see the gitlab issue tracker) history is > > preserved while not forcing anyone to do regular backups. Am I missing > > something? If Mantis cannot be run as readonly, maybe an archived > > version of it [0]? > > We're moving servers right now, so we do need to back up and restore the > database. In case of some hardware failure or accidental deletion, we > don't want to lose the archive. > Raphael is saying, that once Mantis is archived, why would it need to be _regularly_ backed up. It wouldn't. Once it is archived, then we make a backup, and we don't need to make any more. > I'm really confused by the assertion that it's so difficult to migrate > Mantis to GitLab. It might take 1-2 days of work, but in the end it's > just migrating from one SQL schema to the other (and in Mantis' case, > converting structured data into text). > > Admittedly Mantis' schema isn't pretty > (https://github.com/mantisbt/mantisbt/blob/master/admin/schema.php), but > I don't see any fundamental difficulties here at all. > Can you help with this? [...] > > I have a minor doubt regarding wording: is everyone aware that there are > > *pull requests* and *merge requests* in gitlab? > > They're exactly the same [1], at least according to GitLab ;-) > True, but the distinction between forking and doing a merge/pull request from another repo, and doing a merge request from a branch on the same repo is an important distiction to make. > - Florian > > [1] https://about.gitlab.com/2014/09/29/gitlab-flow/ -- "Tools such as > GitHub and Bitbucket choose the name pull request since the first manual > action would be to pull the feature branch. Tools such as GitLab and > Gitorious choose the name merge request since that is the final action > that is requested of the assignee. In this article we'll refer to them > as merge requests." > > ___ > GNUnet-developers mailing list > GNUnet-developers@gnu.org > https://lists.gnu.org/mailman/listinfo/gnunet-developers signature.asc Description: PGP signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
On 4/10/19 1:10 AM, Raphael Arias wrote: > I don't see why the database of a read-only legacy system would need to > be regularly backed up. If it remains online purely as an archive > (possibly with a note to go see the gitlab issue tracker) history is > preserved while not forcing anyone to do regular backups. Am I missing > something? If Mantis cannot be run as readonly, maybe an archived > version of it [0]? We're moving servers right now, so we do need to back up and restore the database. In case of some hardware failure or accidental deletion, we don't want to lose the archive. I'm really confused by the assertion that it's so difficult to migrate Mantis to GitLab. It might take 1-2 days of work, but in the end it's just migrating from one SQL schema to the other (and in Mantis' case, converting structured data into text). Admittedly Mantis' schema isn't pretty (https://github.com/mantisbt/mantisbt/blob/master/admin/schema.php), but I don't see any fundamental difficulties here at all. > I would like to add my voice to Martin's regarding usage of gitlab. I agree with almost everything you're saying. As I've said before, pull requests and code review are great, but overkill for smaller/trivial commits in very small teams. I've also worked in a team before where everything was based on code reviews, even trivial one character changes. It was a pleasure! But the team was rather large, and the tooling had a whole org with multiple teams behind it to make sure it's a good experience. GitLab of course helps compensate for the latter. > I have a minor doubt regarding wording: is everyone aware that there are > *pull requests* and *merge requests* in gitlab? They're exactly the same [1], at least according to GitLab ;-) - Florian [1] https://about.gitlab.com/2014/09/29/gitlab-flow/ -- "Tools such as GitHub and Bitbucket choose the name pull request since the first manual action would be to pull the feature branch. Tools such as GitLab and Gitorious choose the name merge request since that is the final action that is requested of the assignee. In this article we'll refer to them as merge requests." ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
Hi, On 08.04.19 16:12, Christian Grothoff wrote: > On 4/8/19 3:37 PM, Schanzenbach, Martin wrote: >> No, we don't. We dvn et al are faced with unreasonable requirements for the >> use of gitlab which include: >> >> - Migration of Mantis issues -> completely unnecessary. Mantis could remain >> read-only for the "legacy" issues and gitlab used for new issues. > Makes sense, except then Mantis has to remain. That means we have to > keep securing the installation, backing up the database, etc. For how > long? How well will this be done for a legacy system that is rarely > used? Just today I got a report on libmicrohttpd that related to a #32xx > bug which, when re-reading the ancient bug, helped me understand why the > code was written the way it was written. Loosing this memory would be a > major loss, likely more significant than loosing the commit history! So > I think some effort to try to preserve it properly is worth it. And > again, Gitlab was primarily proposed as "better CI", so if its > introduction has other costs, it makes sense to try to minimize those, > right? > I don't see why the database of a read-only legacy system would need to be regularly backed up. If it remains online purely as an archive (possibly with a note to go see the gitlab issue tracker) history is preserved while not forcing anyone to do regular backups. Am I missing something? If Mantis cannot be run as readonly, maybe an archived version of it [0]? I would like to add my voice to Martin's regarding usage of gitlab. I've been extensively using gitlab at work for the last couple of years and it is *really* powerful. I get that GNUnet's main purpose of migrating to it is its CI which is really configurable; if your branch/commit introduces changes that require a change to how the CI behaves, then you just change the .gitlab-ci.yml along with your other changes and the CI will run jobs according to those changes. I'm not really familiar with the current tooling GNUnet uses but at least I remember, coming from Jenkins, Gitlab CI was heaven. I have a minor doubt regarding wording: is everyone aware that there are *pull requests* and *merge requests* in gitlab? The -- in my opinion -- most common workflow in gitlab uses *merge requests* where you develop inside a feature branch in the main repository and then create a merge request, requesting to merge your branch into master. The repository can be arbitrarily configured to - allow a specific set of people to merge - only allow merging if CI passes - only allow merging when a certain number of approvals are given - only allow merging once all discussions are resolved - only allow merging when the branch is rebased on top of the target (to allow fast-forward merges or just a clean history) - create a fast-forward merge or create a merge commit - (I might be missing something) ... As I said, that is configurable, so if it's undesirable to forcibly require two approvals by maintainers or core devs, you don't need to enable that option. But this process is incredibly helpful to review the scope of the whole changeset. I often review my own MR before asking colleagues to do so. I *don't think* it is a good idea to generally *push directly to master*. That should happen only in very few exceptional cases. Anyway, yes there are also forking and pull requests but tbh I have not used those features and for day-to-day development by the "core contributors" that already have git access on gitolite currently, I don't think forking and pull requests should be the workflow. Granted, for people who are new to the project and want to make some changes in their own repo first, it's still a nice feature to have. Hope those comments help. Cheers, R [0] https://www.archive-it.org/ ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
On 4/8/19 4:42 PM, n...@n0.is wrote: >> On 4/8/19 3:37 PM, Schanzenbach, Martin wrote: It sounds like you're suggesting that we should have a core team of developers in official capacity for GNUnet e.V. to look at pull requests and then say "we think that this doesn't infringe on copyright" and merge them in. Is that what you're saying? >>> I did not start the copyright argument and am not even sure if we need to >>> address it (see other mails). >>> What I am saying is that GNUnet e.V. is currently (or better: should be) >>> vetting every contributor wrt the CAA _before_ any contribution is done. >> True. >> >>> This vetting process is not transparent and power is quite concentrated >>> (note I am not saying abused). >> I'm not sure how this is not transparent, as the list of people who >> signed it is maintained in the gnunet-ev.git, and the CAA is public as >> well. > I had to ask where this list is. So I have this 1/4 finished document > which is not a good on-boarding document, but a better one than now, > which is nix, nada. So we should mention little details like this on > the website or in this guide. Sure. >> Also, various people are in principle able to onboard new >> committers. The fact that I collect the printouts is something I'm not >> sure how to fix. I considered putting the signed statements into a Git, >> but figured having scans of people's signatures online was a bad idea (TM). > I remember having to sign a similar document for Erlang contributions, > but they abstracted it into their github-centric organization, so > to have it only digital should be doable if Erlang gets away with it > (also a european business, located in Sweden). Well, for the legality of the CAA, I suspect that's true. As for preventing someone from abusing such a Git as a "free public file hoster" I'm not so sure. Especially stuff in some random developer branch is unlikely to be properly monitored by us. But that could maybe be addressed by a CI check for the upload of 'large' files that triggers an alert. signature.asc Description: OpenPGP digital signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
Christian Grothoff transcribed 8.1K bytes: > On 4/8/19 3:37 PM, Schanzenbach, Martin wrote: > >> It sounds like you're suggesting that we should have a core team of > >> developers in official capacity for GNUnet e.V. to look at pull requests > >> and then say "we think that this doesn't infringe on copyright" and > >> merge them in. Is that what you're saying? > > > > I did not start the copyright argument and am not even sure if we need to > > address it (see other mails). > > What I am saying is that GNUnet e.V. is currently (or better: should be) > > vetting every contributor wrt the CAA _before_ any contribution is done. > > True. > > > This vetting process is not transparent and power is quite concentrated > > (note I am not saying abused). > > I'm not sure how this is not transparent, as the list of people who > signed it is maintained in the gnunet-ev.git, and the CAA is public as > well. I had to ask where this list is. So I have this 1/4 finished document which is not a good on-boarding document, but a better one than now, which is nix, nada. So we should mention little details like this on the website or in this guide. > Also, various people are in principle able to onboard new > committers. The fact that I collect the printouts is something I'm not > sure how to fix. I considered putting the signed statements into a Git, > but figured having scans of people's signatures online was a bad idea (TM). I remember having to sign a similar document for Erlang contributions, but they abstracted it into their github-centric organization, so to have it only digital should be doable if Erlang gets away with it (also a european business, located in Sweden). > > And a "secretary" which is the only entity able to conclude the onboarding > > process is, by definition, an authority. > > True. Nobody claimed this was perfect. > > > So claiming that a restrictive fork+pull policy where members / devs > > sign-off commits hurts the open spirit of the project is just silly. > > It is not getting any more restrictive and bureaucratic than it is now. > > I fail to see how a one-time onboarding issue even relates to something > that would apply to every commit. > > >> I thought we were having an interesting discussion here. You're right, > >> nobody is forcing you to commit to master regularly. That's fine, > >> everybody should use a workflow they're comfortable with. > > > > No, we don't. We dvn et al are faced with unreasonable requirements for the > > use of gitlab which include: > > > > - Migration of Mantis issues -> completely unnecessary. Mantis could remain > > read-only for the "legacy" issues and gitlab used for new issues. > > Makes sense, except then Mantis has to remain. That means we have to > keep securing the installation, backing up the database, etc. For how > long? How well will this be done for a legacy system that is rarely > used? Just today I got a report on libmicrohttpd that related to a #32xx > bug which, when re-reading the ancient bug, helped me understand why the > code was written the way it was written. Loosing this memory would be a > major loss, likely more significant than loosing the commit history! So > I think some effort to try to preserve it properly is worth it. And > again, Gitlab was primarily proposed as "better CI", so if its > introduction has other costs, it makes sense to try to minimize those, > right? > > > - No user forks, no pull requests -> No usability gain over gitolite > > I don't recall anyone saying that. The argument was to not force pull > requests on people. And I don't think 'no user forks' was ever said (we > discussed namespaces for that, right?). Or maybe I'm not understanding. > > > - No automatic user onboarding > The CAA could be included in a pull > > request to the main repo btw. In combination with signed requests this > > would suffice. Also, forks are not touched by the CAA while a push into the > > main repo is. So the entry barrier is much much higher for initial > > contributors. > > Ah, but that's now a suggestion I hear for the first time. I'm not sure > what the lawyers will say about this, but if we can have scans of CAAs > collected by Gitlab (and ideally secured so that the signatures are not > out in the open), I'm all for this. That could be a process improvement, > as long as we can get it done in a way that makes the lawyers happy. > > > All in all I fear this project is a really good idea but doomed from the > > start. > > Using gitlab only because of its CI will just not be good enough and just > > adds another (quite large) tool where the most of the useful functionality > > is unused. > > Well, that was always my concern: that we don't _need_ much of the > functionality as we already have solutions for that. :-) > > > If we use it, we need to embrace its full value offering and see it as a > > change to improve our (CAA) processes. > > That is something I could embrace, but I don't
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
> No, we don't. We dvn et al are faced with unreasonable requirements for the > use of gitlab which include: > > - Migration of Mantis issues -> completely unnecessary. Mantis could remain > read-only for the "legacy" issues and gitlab used for new issues. > - No user forks, no pull requests -> No usability gain over gitolite > - No automatic user onboarding > The CAA could be included in a pull request > to the main repo btw. In combination with signed requests this would suffice. > Also, forks are not touched by the CAA while a push into the main repo is. So > the entry barrier is much much higher for initial contributors. Okay so let's first clear up an apparent misunderstanding. I am not advocating for these restrictions. In fact I think that pull requests on GitLab are a *great* workflow for completely new or infrequent contributors. In fact, there are some more complex changes where I'd be happy to use a pull request and get some feedback first. But I don't want to create a pull request for fixing some typo in a comment somewhere. But for somebody who's through the CAA bureaucracy, I don't see why we should *require* every change to go through a manual pull request with 1-2 sign-offs. They can just push to master, with some automated checks in place. GitLab doesn't prevent this in any way. It doesn't prevent whoever wants to do a pull request to submit one. (For mantis, let's try to have a best-effort migration at least. I also think that user sign-ups should be approved by someone from some admin group, otherwise this is ripe for abuse, some anonymous people will just use gitlab.gnunet.org as their file hosting service otherwise.) > All in all I fear this project is a really good idea but doomed from the > start. > Using gitlab only because of its CI will just not be good enough and just > adds another (quite large) tool where the most of the useful functionality is > unused. > If we use it, we need to embrace its full value offering and see it as a > change to improve our (CAA) processes. > > To me, this has practical impact: > I regularly have students which work on projects wrt GNUnet. > While I would like them to work on it directly, this is completely > unreasonable because of the CAA and the fact that GNUnet tooling is not good > (I am trying not to use strong words here...). > There is _no_ gain from using the GNUnet repos and services. On the contrary! > No CI, no issue integration into commits, a single repo littered with > branches. An issue tracker from the 90's with a gazillion entries to fill in. > Hence, I usually tell them to fork it and work in private on it on a gitlab > instance. > After the work is done (and successful) I may convince them to merge the code > (and sign the CAA). Yes, I get it. Thanks for writing these down in detail. I'm definitely willing to give up some of my "admin conveniences" that gitolite provides to make the developer experience better. I have some technical objections to GitLab vs. gitolite, but I can live with them. - Florian signature.asc Description: OpenPGP digital signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
On 4/8/19 3:37 PM, Schanzenbach, Martin wrote: >> It sounds like you're suggesting that we should have a core team of >> developers in official capacity for GNUnet e.V. to look at pull requests >> and then say "we think that this doesn't infringe on copyright" and >> merge them in. Is that what you're saying? > > I did not start the copyright argument and am not even sure if we need to > address it (see other mails). > What I am saying is that GNUnet e.V. is currently (or better: should be) > vetting every contributor wrt the CAA _before_ any contribution is done. True. > This vetting process is not transparent and power is quite concentrated (note > I am not saying abused). I'm not sure how this is not transparent, as the list of people who signed it is maintained in the gnunet-ev.git, and the CAA is public as well. Also, various people are in principle able to onboard new committers. The fact that I collect the printouts is something I'm not sure how to fix. I considered putting the signed statements into a Git, but figured having scans of people's signatures online was a bad idea (TM). > And a "secretary" which is the only entity able to conclude the onboarding > process is, by definition, an authority. True. Nobody claimed this was perfect. > So claiming that a restrictive fork+pull policy where members / devs sign-off > commits hurts the open spirit of the project is just silly. > It is not getting any more restrictive and bureaucratic than it is now. I fail to see how a one-time onboarding issue even relates to something that would apply to every commit. >> I thought we were having an interesting discussion here. You're right, >> nobody is forcing you to commit to master regularly. That's fine, >> everybody should use a workflow they're comfortable with. > > No, we don't. We dvn et al are faced with unreasonable requirements for the > use of gitlab which include: > > - Migration of Mantis issues -> completely unnecessary. Mantis could remain > read-only for the "legacy" issues and gitlab used for new issues. Makes sense, except then Mantis has to remain. That means we have to keep securing the installation, backing up the database, etc. For how long? How well will this be done for a legacy system that is rarely used? Just today I got a report on libmicrohttpd that related to a #32xx bug which, when re-reading the ancient bug, helped me understand why the code was written the way it was written. Loosing this memory would be a major loss, likely more significant than loosing the commit history! So I think some effort to try to preserve it properly is worth it. And again, Gitlab was primarily proposed as "better CI", so if its introduction has other costs, it makes sense to try to minimize those, right? > - No user forks, no pull requests -> No usability gain over gitolite I don't recall anyone saying that. The argument was to not force pull requests on people. And I don't think 'no user forks' was ever said (we discussed namespaces for that, right?). Or maybe I'm not understanding. > - No automatic user onboarding > The CAA could be included in a pull request > to the main repo btw. In combination with signed requests this would suffice. > Also, forks are not touched by the CAA while a push into the main repo is. So > the entry barrier is much much higher for initial contributors. Ah, but that's now a suggestion I hear for the first time. I'm not sure what the lawyers will say about this, but if we can have scans of CAAs collected by Gitlab (and ideally secured so that the signatures are not out in the open), I'm all for this. That could be a process improvement, as long as we can get it done in a way that makes the lawyers happy. > All in all I fear this project is a really good idea but doomed from the > start. > Using gitlab only because of its CI will just not be good enough and just > adds another (quite large) tool where the most of the useful functionality is > unused. Well, that was always my concern: that we don't _need_ much of the functionality as we already have solutions for that. :-) > If we use it, we need to embrace its full value offering and see it as a > change to improve our (CAA) processes. That is something I could embrace, but I don't know the Gitlab capabilities here. Can we collect a digital signature (as in, a scan?). Can we keep those securely? > To me, this has practical impact: > I regularly have students which work on projects wrt GNUnet. > While I would like them to work on it directly, this is completely > unreasonable because of the CAA and the fact that GNUnet tooling is not good > (I am trying not to use strong words here...). I think you need to have some _real_ paperwork in your life sometimes if the single-signature CAA is "completely unreasonable". > There is _no_ gain from using the GNUnet repos and services. On the contrary! > No CI, no issue integration into commits, a single repo littered with > branches. An issue
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
> On 7. Apr 2019, at 19:20, Florian Dold wrote: > > On 4/7/19 6:46 PM, Schanzenbach, Martin wrote: >> The CAA does not help in any way. You are still liable as a platform. It has >> literally nothing to do with the copyright infringements if the contributor >> copied code from somewhere else. You cannot delegate this responsibility >> anymore to the user. That way the old way of doing it. > > I am not sure what requirements you are talking about. Do you have some > reference that explains this? Christian, do you have one? I would > guess we are probably not the only project affected by this. > > It sounds like you're suggesting that we should have a core team of > developers in official capacity for GNUnet e.V. to look at pull requests > and then say "we think that this doesn't infringe on copyright" and > merge them in. Is that what you're saying? I did not start the copyright argument and am not even sure if we need to address it (see other mails). What I am saying is that GNUnet e.V. is currently (or better: should be) vetting every contributor wrt the CAA _before_ any contribution is done. This vetting process is not transparent and power is quite concentrated (note I am not saying abused). And a "secretary" which is the only entity able to conclude the onboarding process is, by definition, an authority. So claiming that a restrictive fork+pull policy where members / devs sign-off commits hurts the open spirit of the project is just silly. It is not getting any more restrictive and bureaucratic than it is now. > > If somebody hosts code on the GNUnet gitlab that infringes copyright, > wouldn't we also be responsible for this? How does the pull request > policy solve that? > >> Just do whatever you want. I think I will stop bringing myself in this >> regard and just develop in an external mirror repo occasionally rebasing my >> stuff in the main repo. This way, you can even keep gitolite -> win win > > I thought we were having an interesting discussion here. You're right, > nobody is forcing you to commit to master regularly. That's fine, > everybody should use a workflow they're comfortable with. No, we don't. We dvn et al are faced with unreasonable requirements for the use of gitlab which include: - Migration of Mantis issues -> completely unnecessary. Mantis could remain read-only for the "legacy" issues and gitlab used for new issues. - No user forks, no pull requests -> No usability gain over gitolite - No automatic user onboarding > The CAA could be included in a pull request to the main repo btw. In combination with signed requests this would suffice. Also, forks are not touched by the CAA while a push into the main repo is. So the entry barrier is much much higher for initial contributors. All in all I fear this project is a really good idea but doomed from the start. Using gitlab only because of its CI will just not be good enough and just adds another (quite large) tool where the most of the useful functionality is unused. If we use it, we need to embrace its full value offering and see it as a change to improve our (CAA) processes. To me, this has practical impact: I regularly have students which work on projects wrt GNUnet. While I would like them to work on it directly, this is completely unreasonable because of the CAA and the fact that GNUnet tooling is not good (I am trying not to use strong words here...). There is _no_ gain from using the GNUnet repos and services. On the contrary! No CI, no issue integration into commits, a single repo littered with branches. An issue tracker from the 90's with a gazillion entries to fill in. Hence, I usually tell them to fork it and work in private on it on a gitlab instance. After the work is done (and successful) I may convince them to merge the code (and sign the CAA). > > - Florian > signature.asc Description: Message signed with OpenPGP ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
On 4/8/19 12:33 PM, Marcos Marado wrote: > I am assuming that the "EU regulation" here spoken of is the Copyright Reform, > in particular Article 13. If that's the case, then “open source > software development > and sharing platforms” are explicitly excluded from it[1] (let's see > how will that be > transposed into national laws). It is arguable what defines something > as being an > "OSS dev & share platform", but I think it is fair to assume that a > gitlab instance > run and maintained by GNUnet e.V. and with the purpose of hosting only free > software would qualify. I hope so, but a public Git can in principle be used for anything. I've been storing movies in Git repos (yes, I'm insane). So just running Gitlab probably doesn't qualify for the exception. But if we have the CAA which makes it clear that this Git is about Free Software development, we're likely in the clear (but I'll never declare anything a certainty these days). And sure, the caveats others mentioned here about this not yet being national law apply, but this is a forward-looking discussion. signature.asc Description: OpenPGP digital signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
Hi, On Mon, Apr 8, 2019 at 1:07 PM wrote: [...] > > I am assuming that the "EU regulation" here spoken of is the Copyright > > Reform, > > in particular Article 13. If that's the case, then “open source > > software development > > and sharing platforms” are explicitly excluded from it[1] (let's see > > how will that be > > transposed into national laws). It is arguable what defines something > > as being an > > "OSS dev & share platform", but I think it is fair to assume that a > > gitlab instance > > run and maintained by GNUnet e.V. and with the purpose of hosting only free > > software would qualify. [...] > > If the OSS exemption didn't exist, this wouldn't be enough: those > > within the core > > team would need the tools to validate copyright infringement on each commit, > > instead of a simple "I looked at it and, after a cursory glance, I > > don't think it > > infringes any copyright". > > This sounds pretty much like something EU should pay for, because > without being able to go into details (for obvious reasons), the ways > to analyze code out there in a way which satisfies the industry > are endless and require almost as much resources. > Imo we simply don't have the resources to fullfil this in GNUnet > unless someone comes up with something pre-made we can use. To > be quiet honest, this is a problem which affects Free and Open Source > software at large, so to expect a complete check of every line of > code for possible infringement must be automated. I don't want > us to grow but at the same time slow down with each and every new > committer. Well... as I said, the eV will be exempt if the gitlab instance has, as I assume it does, the sole purpose of hosting free software. But yes, despite having managed to create that exemption, Article 13 is still harmful to free software (and the society in general), and this is something EU should not have accepted. Unfortunately, that was not the case*. * well, in theory there is still a chance that this gets rejected: there is going to be a vote by the Council of the European Union next monday (15th April), and it could still be rejected if Germany voted against it -- which doesn't seem to be something they're willing to do. -- Marcos Marado ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
Marcos Marado transcribed 2.6K bytes: > Hi, > > On Sun, Apr 7, 2019, 18:20 Florian Dold wrote: > > On 4/7/19 6:46 PM, Schanzenbach, Martin wrote: > > > The CAA does not help in any way. You are still liable as a platform. It > > > has literally nothing to do with the copyright infringements if the > > > contributor copied code from somewhere else. You cannot delegate this > > > responsibility anymore to the user. That way the old way of doing it. > > > > I am not sure what requirements you are talking about. Do you have some > > reference that explains this? Christian, do you have one? I would > > guess we are probably not the only project affected by this. > > I am assuming that the "EU regulation" here spoken of is the Copyright Reform, > in particular Article 13. If that's the case, then “open source > software development > and sharing platforms” are explicitly excluded from it[1] (let's see > how will that be > transposed into national laws). It is arguable what defines something > as being an > "OSS dev & share platform", but I think it is fair to assume that a > gitlab instance > run and maintained by GNUnet e.V. and with the purpose of hosting only free > software would qualify. > > > It sounds like you're suggesting that we should have a core team of > > developers in official capacity for GNUnet e.V. to look at pull requests > > and then say "we think that this doesn't infringe on copyright" and > > merge them in. Is that what you're saying? > > If the OSS exemption didn't exist, this wouldn't be enough: those > within the core > team would need the tools to validate copyright infringement on each commit, > instead of a simple "I looked at it and, after a cursory glance, I > don't think it > infringes any copyright". This sounds pretty much like something EU should pay for, because without being able to go into details (for obvious reasons), the ways to analyze code out there in a way which satisfies the industry are endless and require almost as much resources. Imo we simply don't have the resources to fullfil this in GNUnet unless someone comes up with something pre-made we can use. To be quiet honest, this is a problem which affects Free and Open Source software at large, so to expect a complete check of every line of code for possible infringement must be automated. I don't want us to grow but at the same time slow down with each and every new committer. I do get that the eV is responsible, but there ought to be a different way. > [1] > http://www.openforumeurope.org/copyright-directive-approved-with-art-11-13/ > > Best regards, > -- > Marcos Marado > > ___ > GNUnet-developers mailing list > GNUnet-developers@gnu.org > https://lists.gnu.org/mailman/listinfo/gnunet-developers ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
z...@hyper.dev transcribed 462 bytes: > Hello! > > On 2019-04-05 21:20, Devan C. dvn wrote: > > Hello my fellow GNUnetians, > > > > > - Registration is open. There are no guarantees on uptime, or even > > data retention (though I don't expect data to disappear). > > Does it mean I can register an account and use it to temporarily host my > project? I don't think so, or I am not sure. We are still discussing final details about what we are able to do (see rest of the thread). Once this is done, we should have a more specific idea of what we may and may not do as association and as individuals. > ___ > GNUnet-developers mailing list > GNUnet-developers@gnu.org > https://lists.gnu.org/mailman/listinfo/gnunet-developers > ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
Hi, On Sun, Apr 7, 2019, 18:20 Florian Dold wrote: > On 4/7/19 6:46 PM, Schanzenbach, Martin wrote: > > The CAA does not help in any way. You are still liable as a platform. It > > has literally nothing to do with the copyright infringements if the > > contributor copied code from somewhere else. You cannot delegate this > > responsibility anymore to the user. That way the old way of doing it. > > I am not sure what requirements you are talking about. Do you have some > reference that explains this? Christian, do you have one? I would > guess we are probably not the only project affected by this. I am assuming that the "EU regulation" here spoken of is the Copyright Reform, in particular Article 13. If that's the case, then “open source software development and sharing platforms” are explicitly excluded from it[1] (let's see how will that be transposed into national laws). It is arguable what defines something as being an "OSS dev & share platform", but I think it is fair to assume that a gitlab instance run and maintained by GNUnet e.V. and with the purpose of hosting only free software would qualify. > It sounds like you're suggesting that we should have a core team of > developers in official capacity for GNUnet e.V. to look at pull requests > and then say "we think that this doesn't infringe on copyright" and > merge them in. Is that what you're saying? If the OSS exemption didn't exist, this wouldn't be enough: those within the core team would need the tools to validate copyright infringement on each commit, instead of a simple "I looked at it and, after a cursory glance, I don't think it infringes any copyright". [1] http://www.openforumeurope.org/copyright-directive-approved-with-art-11-13/ Best regards, -- Marcos Marado ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
Hello! On 2019-04-05 21:20, Devan C. dvn wrote: Hello my fellow GNUnetians, - Registration is open. There are no guarantees on uptime, or even data retention (though I don't expect data to disappear). Does it mean I can register an account and use it to temporarily host my project? ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
On 4/7/19 6:46 PM, Schanzenbach, Martin wrote: > The CAA does not help in any way. You are still liable as a platform. It has > literally nothing to do with the copyright infringements if the contributor > copied code from somewhere else. You cannot delegate this responsibility > anymore to the user. That way the old way of doing it. I am not sure what requirements you are talking about. Do you have some reference that explains this? Christian, do you have one? I would guess we are probably not the only project affected by this. It sounds like you're suggesting that we should have a core team of developers in official capacity for GNUnet e.V. to look at pull requests and then say "we think that this doesn't infringe on copyright" and merge them in. Is that what you're saying? If somebody hosts code on the GNUnet gitlab that infringes copyright, wouldn't we also be responsible for this? How does the pull request policy solve that? > Just do whatever you want. I think I will stop bringing myself in this regard > and just develop in an external mirror repo occasionally rebasing my stuff in > the main repo. This way, you can even keep gitolite -> win win I thought we were having an interesting discussion here. You're right, nobody is forcing you to commit to master regularly. That's fine, everybody should use a workflow they're comfortable with. - Florian signature.asc Description: OpenPGP digital signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
> On 7. Apr 2019, at 13:36, Christian Grothoff wrote: > > On 4/7/19 11:11 AM, Schanzenbach, Martin wrote: On 4/7/19 8:33 AM, Schanzenbach, Martin wrote: > Contributors should be able to do anything they want in their own > namespaces including committing code that does not compile (e.g. for > their gnunet.git forks). However, in order to get it into the "main" > gnunet project codebase, the CI must pass for the respective pull > request and I would argue that 1-2 "main" devs should sign off on the > commit (this allows us to control the CAA issue a bit). Eh, sorry, but under forthcoming EU regulation, we cannot even host contributor's code without having a signed the CAA. So Git pushes should only be possible for people that signed the CAA, and in that case if a CAA-signing contributor has pushed a change to a namespace/branch that by convention is to be merged, we should ideally automate the merge. >>> >>> I think you misunderstand the new regulation. Having a CAA does not protect >>> the platform from this. >>> It is not enough to have the user state that the code is his, the platform >>> must verify/ensure that. >>> No legal document is able to absolve us from this. >> >> Btw I also think it only applied to commercial platforms. So is there really >> a need to worry??? > > A hostile actor can very easily classify you as "commercial", starting > with GNUnet e.V. applying for grants and hiring devs thus potentially > providing income for some people. And as we learned with Telekom, legal > disputes with the wrong party can bankrupt you easily even if you > _would_ win it, and the Vorstand has _personal_ liability in case of > gross negligence. IANAL, but I believe the CAA should help us against > any claims of _gross_ negligence. OTOH, I'm sure there will be lawyers > that would twist the situation of allowing anyone to upload any code > without any such contract as gross negligence. The CAA does not help in any way. You are still liable as a platform. It has literally nothing to do with the copyright infringements if the contributor copied code from somewhere else. You cannot delegate this responsibility anymore to the user. That way the old way of doing it. > However, given that we cannot then preserve the gpg signature on the commit (depending on how the merge goes), maybe indeed we _need_ a dev to do the sign-off just to get at least one proper gpg signature on the commit. In that case, maybe the CI can automatically send an e-mail to a group of devs that are on sanity-checking + gpg-signing duty. Anyway, the CAA issue should be solved prior to any Git write access, and the sign off step _may_ be (borderline) acceptable to address the GPG signing issue, but it shouldn't be seen or phrased as that this is done by the "main" devs. The sign-off should be more more like a secretary position for pushing the paperwork along. >>> >>> Well then the whole "open participation" thing is moot anyway and I wonder >>> why it comes up all the time here. >>> If we have a beaurocractic onboarding process including the CAA (which we >>> do not have atm btw), then participation is limited and must be done >>> through gatekeepers anyway. > > I'm not sure why you say "which we do not have atm btw" -- all the > Gitolite admins have been told that giving Git (write) access should > only be done if someone has signed the CAA, and I have a collection of > CAA sheets here at home. I'm not 100% sure that there isn't one missing > (some are electronic, I still need to print those), but at least the > process exists and _should_ be enforced. Oh of course! A transparent, unbureaucratic and open way for contributor onboarding! What can possibly go wrong. "Git admins" instructed by you on the basis of sheets of paper which have been mailed to you. In the year 2019... Just do whatever you want. I think I will stop bringing myself in this regard and just develop in an external mirror repo occasionally rebasing my stuff in the main repo. This way, you can even keep gitolite -> win win > > signature.asc Description: Message signed with OpenPGP ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
On 4/7/19 11:11 AM, Schanzenbach, Martin wrote: >>> On 4/7/19 8:33 AM, Schanzenbach, Martin wrote: Contributors should be able to do anything they want in their own namespaces including committing code that does not compile (e.g. for their gnunet.git forks). However, in order to get it into the "main" gnunet project codebase, the CI must pass for the respective pull request and I would argue that 1-2 "main" devs should sign off on the commit (this allows us to control the CAA issue a bit). >>> Eh, sorry, but under forthcoming EU regulation, we cannot even host >>> contributor's code without having a signed the CAA. So Git pushes should >>> only be possible for people that signed the CAA, and in that case if a >>> CAA-signing contributor has pushed a change to a namespace/branch that >>> by convention is to be merged, we should ideally automate the merge. >> >> I think you misunderstand the new regulation. Having a CAA does not protect >> the platform from this. >> It is not enough to have the user state that the code is his, the platform >> must verify/ensure that. >> No legal document is able to absolve us from this. > > Btw I also think it only applied to commercial platforms. So is there really > a need to worry??? A hostile actor can very easily classify you as "commercial", starting with GNUnet e.V. applying for grants and hiring devs thus potentially providing income for some people. And as we learned with Telekom, legal disputes with the wrong party can bankrupt you easily even if you _would_ win it, and the Vorstand has _personal_ liability in case of gross negligence. IANAL, but I believe the CAA should help us against any claims of _gross_ negligence. OTOH, I'm sure there will be lawyers that would twist the situation of allowing anyone to upload any code without any such contract as gross negligence. >>> However, given that we cannot then preserve the gpg signature on the >>> commit (depending on how the merge goes), maybe indeed we _need_ a dev >>> to do the sign-off just to get at least one proper gpg signature on the >>> commit. In that case, maybe the CI can automatically send an e-mail to >>> a group of devs that are on sanity-checking + gpg-signing duty. >>> >>> Anyway, the CAA issue should be solved prior to any Git write access, >>> and the sign off step _may_ be (borderline) acceptable to address the >>> GPG signing issue, but it shouldn't be seen or phrased as that this is >>> done by the "main" devs. The sign-off should be more more like a >>> secretary position for pushing the paperwork along. >> >> Well then the whole "open participation" thing is moot anyway and I wonder >> why it comes up all the time here. >> If we have a beaurocractic onboarding process including the CAA (which we do >> not have atm btw), then participation is limited and must be done through >> gatekeepers anyway. I'm not sure why you say "which we do not have atm btw" -- all the Gitolite admins have been told that giving Git (write) access should only be done if someone has signed the CAA, and I have a collection of CAA sheets here at home. I'm not 100% sure that there isn't one missing (some are electronic, I still need to print those), but at least the process exists and _should_ be enforced. signature.asc Description: OpenPGP digital signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
On 4/7/19 1:14 PM, Schanzenbach, Martin wrote: > As I said in another mail you are fighting windmills here. Given that we have > a mandatory CAA, that is where the gatekeepers come in anyway. > And the problems you claim here are also exiting there. Signing the CAA isn't really imposing gatekeeping. First of all, it's a one time act, not something for every commit. Second, you can do it by yourself, at best your employer might stop you from signing (and then you could change jobs). And the person checking that the CAA is signed is really just acting as a secretary: there is no brain involved in that process, in theory it could be automated. So that's very, very different from gatekeeping IMO. > And regarding domain knowledge: It doesn't matter. Code review is more than > just functionality. It is QA. Florian's argument on capacity still holds here. And as I keep saying, to avoid mistakes, the best way is to automate the review, that's what CI is all about. And I'm sure about one thing: for certain types of issues, the CI will do a better job than me. And for the _rest_, we do have the gnunet-svn mailinglist which does generally work to provide post-merge feedback, which should suffice IMO. signature.asc Description: OpenPGP digital signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
> On 7. Apr 2019, at 12:47, Florian Dold wrote: > > On 4/7/19 8:33 AM, Schanzenbach, Martin wrote: >> Contributors should be able to do anything they want in their own namespaces >> including committing code that does not compile (e.g. for their gnunet.git >> forks). >> However, in order to get it into the "main" gnunet project codebase, the CI >> must pass for the respective pull request and I would argue that 1-2 "main" >> devs should sign off on the commit (this allows us to control the CAA issue >> a bit). >> Then, things like 0.11.1 and 0.11.2 will not happen anymore and devs still >> have the freedom to commit their current work even if does not compile. > > I'm still not convinced. Everybody can already use their own branches > even right now to commit code that doesn't compile. > > Do we even have enough "main devs" to make it feasible to require 1-2 > gatekeeper sign-offs for every commit? What if somebody is on vacation? > What about experimental subsystems like RPS? Is there anybody else > than grothoff who would have the domain knowledge to sign off commits on > RPS for ch3? As I said in another mail you are fighting windmills here. Given that we have a mandatory CAA, that is where the gatekeepers come in anyway. And the problems you claim here are also exiting there. And regarding domain knowledge: It doesn't matter. Code review is more than just functionality. It is QA. > > I'm worried that this will lead to a balkanization of the project, where > everybody just works on their own branch, because some want to make > integrating changes into master so tedious. It'll also make more > sweeping changes and refactoring much harder to pull off. > > Once we grow really big, we can do all this. Great if we already have > the infrastructure partially in place. Then we can even have some core > repo with a lot of gate keeping. But for the current situation, that's > just overkill and does more harm than good IMHO. We already have and need gatekeeping. And as you already said above, everybody can already work on their own branch. > > GNUnet should be fun and anarchy, not bureaucracy and gatekeeping. > I really don't want to repeat myself but you really don't see the point. > - Florian > signature.asc Description: Message signed with OpenPGP ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
Some more detailed arguments for the point I was making: http://lisperator.net/blog/pull-request-based-development-sucks/ I really like that with GitLab we have to *possibility* to do code review, but we shouldn't force it down everybody's throat for GNUnet. - Florian On 4/7/19 12:47 PM, Florian Dold wrote: > On 4/7/19 8:33 AM, Schanzenbach, Martin wrote: >> Contributors should be able to do anything they want in their own namespaces >> including committing code that does not compile (e.g. for their gnunet.git >> forks). >> However, in order to get it into the "main" gnunet project codebase, the CI >> must pass for the respective pull request and I would argue that 1-2 "main" >> devs should sign off on the commit (this allows us to control the CAA issue >> a bit). >> Then, things like 0.11.1 and 0.11.2 will not happen anymore and devs still >> have the freedom to commit their current work even if does not compile. > > I'm still not convinced. Everybody can already use their own branches > even right now to commit code that doesn't compile. > > Do we even have enough "main devs" to make it feasible to require 1-2 > gatekeeper sign-offs for every commit? What if somebody is on vacation? > What about experimental subsystems like RPS? Is there anybody else > than grothoff who would have the domain knowledge to sign off commits on > RPS for ch3? > > I'm worried that this will lead to a balkanization of the project, where > everybody just works on their own branch, because some want to make > integrating changes into master so tedious. It'll also make more > sweeping changes and refactoring much harder to pull off. > > Once we grow really big, we can do all this. Great if we already have > the infrastructure partially in place. Then we can even have some core > repo with a lot of gate keeping. But for the current situation, that's > just overkill and does more harm than good IMHO. > > GNUnet should be fun and anarchy, not bureaucracy and gatekeeping. > > - Florian > signature.asc Description: OpenPGP digital signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
On 4/7/19 8:33 AM, Schanzenbach, Martin wrote: > Contributors should be able to do anything they want in their own namespaces > including committing code that does not compile (e.g. for their gnunet.git > forks). > However, in order to get it into the "main" gnunet project codebase, the CI > must pass for the respective pull request and I would argue that 1-2 "main" > devs should sign off on the commit (this allows us to control the CAA issue a > bit). > Then, things like 0.11.1 and 0.11.2 will not happen anymore and devs still > have the freedom to commit their current work even if does not compile. I'm still not convinced. Everybody can already use their own branches even right now to commit code that doesn't compile. Do we even have enough "main devs" to make it feasible to require 1-2 gatekeeper sign-offs for every commit? What if somebody is on vacation? What about experimental subsystems like RPS? Is there anybody else than grothoff who would have the domain knowledge to sign off commits on RPS for ch3? I'm worried that this will lead to a balkanization of the project, where everybody just works on their own branch, because some want to make integrating changes into master so tedious. It'll also make more sweeping changes and refactoring much harder to pull off. Once we grow really big, we can do all this. Great if we already have the infrastructure partially in place. Then we can even have some core repo with a lot of gate keeping. But for the current situation, that's just overkill and does more harm than good IMHO. GNUnet should be fun and anarchy, not bureaucracy and gatekeeping. - Florian signature.asc Description: OpenPGP digital signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
> On 7. Apr 2019, at 11:11, Schanzenbach, Martin > wrote: > > > >> On 7. Apr 2019, at 11:02, Christian Grothoff wrote: >> >> On 4/7/19 8:33 AM, Schanzenbach, Martin wrote: >>> Contributors should be able to do anything they want in their own >>> namespaces including committing code that does not compile (e.g. for >>> their gnunet.git forks). However, in order to get it into the "main" >>> gnunet project codebase, the CI must pass for the respective pull >>> request and I would argue that 1-2 "main" devs should sign off on the >>> commit (this allows us to control the CAA issue a bit). >> Eh, sorry, but under forthcoming EU regulation, we cannot even host >> contributor's code without having a signed the CAA. So Git pushes should >> only be possible for people that signed the CAA, and in that case if a >> CAA-signing contributor has pushed a change to a namespace/branch that >> by convention is to be merged, we should ideally automate the merge. > > I think you misunderstand the new regulation. Having a CAA does not protect > the platform from this. > It is not enough to have the user state that the code is his, the platform > must verify/ensure that. > No legal document is able to absolve us from this. Btw I also think it only applied to commercial platforms. So is there really a need to worry??? > >> >> However, given that we cannot then preserve the gpg signature on the >> commit (depending on how the merge goes), maybe indeed we _need_ a dev >> to do the sign-off just to get at least one proper gpg signature on the >> commit. In that case, maybe the CI can automatically send an e-mail to >> a group of devs that are on sanity-checking + gpg-signing duty. >> >> Anyway, the CAA issue should be solved prior to any Git write access, >> and the sign off step _may_ be (borderline) acceptable to address the >> GPG signing issue, but it shouldn't be seen or phrased as that this is >> done by the "main" devs. The sign-off should be more more like a >> secretary position for pushing the paperwork along. > > Well then the whole "open participation" thing is moot anyway and I wonder > why it comes up all the time here. > If we have a beaurocractic onboarding process including the CAA (which we do > not have atm btw), then participation is limited and must be done through > gatekeepers anyway. > OTOH, I do not really see a problem with fork+edit without the CAA. The > problem _only_ arises when the code is merged into the main repo. > Which is why I think my proposal is better. (apart from the EU regulation > stuff, but there is no solution to that) > >> >> WDYT? signature.asc Description: Message signed with OpenPGP ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
> On 7. Apr 2019, at 11:02, Christian Grothoff wrote: > > On 4/7/19 8:33 AM, Schanzenbach, Martin wrote: >> Contributors should be able to do anything they want in their own >> namespaces including committing code that does not compile (e.g. for >> their gnunet.git forks). However, in order to get it into the "main" >> gnunet project codebase, the CI must pass for the respective pull >> request and I would argue that 1-2 "main" devs should sign off on the >> commit (this allows us to control the CAA issue a bit). > Eh, sorry, but under forthcoming EU regulation, we cannot even host > contributor's code without having a signed the CAA. So Git pushes should > only be possible for people that signed the CAA, and in that case if a > CAA-signing contributor has pushed a change to a namespace/branch that > by convention is to be merged, we should ideally automate the merge. I think you misunderstand the new regulation. Having a CAA does not protect the platform from this. It is not enough to have the user state that the code is his, the platform must verify/ensure that. No legal document is able to absolve us from this. > > However, given that we cannot then preserve the gpg signature on the > commit (depending on how the merge goes), maybe indeed we _need_ a dev > to do the sign-off just to get at least one proper gpg signature on the > commit. In that case, maybe the CI can automatically send an e-mail to > a group of devs that are on sanity-checking + gpg-signing duty. > > Anyway, the CAA issue should be solved prior to any Git write access, > and the sign off step _may_ be (borderline) acceptable to address the > GPG signing issue, but it shouldn't be seen or phrased as that this is > done by the "main" devs. The sign-off should be more more like a > secretary position for pushing the paperwork along. Well then the whole "open participation" thing is moot anyway and I wonder why it comes up all the time here. If we have a beaurocractic onboarding process including the CAA (which we do not have atm btw), then participation is limited and must be done through gatekeepers anyway. OTOH, I do not really see a problem with fork+edit without the CAA. The problem _only_ arises when the code is merged into the main repo. Which is why I think my proposal is better. (apart from the EU regulation stuff, but there is no solution to that) > > WDYT? > signature.asc Description: Message signed with OpenPGP ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
On 4/7/19 8:33 AM, Schanzenbach, Martin wrote: > Contributors should be able to do anything they want in their own > namespaces including committing code that does not compile (e.g. for > their gnunet.git forks). However, in order to get it into the "main" > gnunet project codebase, the CI must pass for the respective pull > request and I would argue that 1-2 "main" devs should sign off on the > commit (this allows us to control the CAA issue a bit). Eh, sorry, but under forthcoming EU regulation, we cannot even host contributor's code without having a signed the CAA. So Git pushes should only be possible for people that signed the CAA, and in that case if a CAA-signing contributor has pushed a change to a namespace/branch that by convention is to be merged, we should ideally automate the merge. However, given that we cannot then preserve the gpg signature on the commit (depending on how the merge goes), maybe indeed we _need_ a dev to do the sign-off just to get at least one proper gpg signature on the commit. In that case, maybe the CI can automatically send an e-mail to a group of devs that are on sanity-checking + gpg-signing duty. Anyway, the CAA issue should be solved prior to any Git write access, and the sign off step _may_ be (borderline) acceptable to address the GPG signing issue, but it shouldn't be seen or phrased as that this is done by the "main" devs. The sign-off should be more more like a secretary position for pushing the paperwork along. WDYT? signature.asc Description: OpenPGP digital signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
> On 6. Apr 2019, at 21:47, Florian Dold wrote: > > Signed PGP part > Thanks for taking the time to set this up. So far some things don't > seem right yet: > > There is a massive security problem. Everybody (!!) is able to create > accounts and set their password, *without* being the owner of the > respective email address. As "proof", I've been so friendly to create > an account and sample project "as Christian" (sorry Christian!). > > https://gitlab.gnunet.org/grothoff/gitlab-is-so-awesome-but-insecure > > Note that this account has Christian's email address associated with it > (which I obviously don't control), but I was able to set his password. > There was no email confirmation step, like there usually is with most > other platforms. This is, eh, not great. I can sign up anybody else, > they won't get a confirmation. > > (Of course anybody can create an account with a fake name and email > address, but I would expect that you can only log in after you've > confirmed that you CONTROL that email address.) > > Some other small things: > * resources (images) are missing because of some bad / misconfigured > HTTP header (see below) > * when I go to gitlab.gnunet.org, it asks me for a login. instead it > should show me the list of projects > * even when I click on "Explore" in the footer, it shows me an empty > list of "trending repositories", so the actual list of repositories is > two clicks away from the landing page. > > And a more general comment: Having some CI bot that rejects bad commits > would be great. But I'd rather dislike if we would define a bunch of > gatekeepers who have to approve merge request from contributors. So I'd > prefer if we were liberal with giving access to the main gnunet repo, > and not create some heavy gatekeeping policies. Contributors should be able to do anything they want in their own namespaces including committing code that does not compile (e.g. for their gnunet.git forks). However, in order to get it into the "main" gnunet project codebase, the CI must pass for the respective pull request and I would argue that 1-2 "main" devs should sign off on the commit (this allows us to control the CAA issue a bit). Then, things like 0.11.1 and 0.11.2 will not happen anymore and devs still have the freedom to commit their current work even if does not compile. > > - Florian > > > """ > Error parsing header X-XSS-Protection: 1; mode=block, 1; mode=block: > expected semicolon at character position 13. The default protections > will be applied. > > Refused to load the image '' because it violates the following > Content Security Policy directive: "default-src 'self'". Note that > 'img-src' was not explicitly set, so 'default-src' is used as a fallback. > > document-register-element.node.js:1296 [Deprecation] > document.registerElement is deprecated and will be removed in M73, > around March 2019. Please use window.customElements.define instead. See > https://www.chromestatus.com/features/4642138092470272 for more details. > """ > > On 4/5/19 9:20 PM, Devan C. dvn wrote: >> Hello my fellow GNUnetians, >> >> As some of you know, I have been pushing for and working on getting us >> to CI/CD system based on Gitlab CI. This is pretty much ready to start >> using and building out the pipelines for now. >> >> In a previous thread[0] there was a decision to give Gitlab a try, as >> more than just our CI system, and to migrate our repos from Gitolite to >> our own Gitlab instance. >> There was less agreement regarding the idea to move from MantisBT to >> Gitlab's issue tracker, but in the end we decided to try importing all >> of the >2k Mantis tickets into Gitlab.[1] >> >> This email is a status update on the overall progress of moving to >> Gitlab. >> >> Current Gitlab status: >> >> - Gitlab is running and accessible at `https://gitlab.gnunet.org` >> >> - Registration is open. There are no guarantees on uptime, or even >> data retention (though I don't expect data to disappear). >> >> - wldhx has kindly offered two "Gitlab Runners" for running CI jobs. >> These will be added as shared runners, to be used by any projects on >> the instance. This may be changed later to only be shared by projects >> under the GNUnet namespace. >> >> **TODO** >> >> - Setup email. Used for registration, password resets, >> notifcations, and interaction (eg. issue threads). >> >> - Currently run using containers with docker-compose. Will switch to >> using systemd services to with the containers, removing docker-compose. >> >> - Daily remote backups. Perhaps 'firefly.gnunet.org' could be the backup >> site, hmm? >> >> - Change configured hostname (in Gitlab) to 'gitlab.gnunet.org'. >> >> - Setup redirect from 'git.gnunet.org' -> 'gitlab.gnunet.org' >> >> >> Current [MantisBT -> Gitlab Issues] status: >> >> Exporting: >> - Mantis can export to CSV and "Excell" XML >> - These do not contain comments (bugnotes). It looks like there might >>be a
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
On 4/6/19 9:47 PM, Florian Dold wrote: > Error parsing header X-XSS-Protection: 1; mode=block, 1; mode=block: > expected semicolon at character position 13. The default protections > will be applied. I've fixed this, seems Gitlab is adding this, and the Apache config was _also_ adding it, resulting in the header being added twice. > Refused to load the image '' because it violates the following > Content Security Policy directive: "default-src 'self'". Note that > 'img-src' was not explicitly set, so 'default-src' is used as a fallback. I've added "data:" to the acceptable img-src'es now. signature.asc Description: OpenPGP digital signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
On 4/6/19 9:47 PM, Florian Dold wrote: > Thanks for taking the time to set this up. So far some things don't > seem right yet: > > There is a massive security problem. Everybody (!!) is able to create > accounts and set their password, *without* being the owner of the > respective email address. As "proof", I've been so friendly to create > an account and sample project "as Christian" (sorry Christian!). > > https://gitlab.gnunet.org/grothoff/gitlab-is-so-awesome-but-insecure I can confirm the hack. Nice job! :-) Go gitlab. Secure by default. signature.asc Description: OpenPGP digital signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
Thanks for taking the time to set this up. So far some things don't seem right yet: There is a massive security problem. Everybody (!!) is able to create accounts and set their password, *without* being the owner of the respective email address. As "proof", I've been so friendly to create an account and sample project "as Christian" (sorry Christian!). https://gitlab.gnunet.org/grothoff/gitlab-is-so-awesome-but-insecure Note that this account has Christian's email address associated with it (which I obviously don't control), but I was able to set his password. There was no email confirmation step, like there usually is with most other platforms. This is, eh, not great. I can sign up anybody else, they won't get a confirmation. (Of course anybody can create an account with a fake name and email address, but I would expect that you can only log in after you've confirmed that you CONTROL that email address.) Some other small things: * resources (images) are missing because of some bad / misconfigured HTTP header (see below) * when I go to gitlab.gnunet.org, it asks me for a login. instead it should show me the list of projects * even when I click on "Explore" in the footer, it shows me an empty list of "trending repositories", so the actual list of repositories is two clicks away from the landing page. And a more general comment: Having some CI bot that rejects bad commits would be great. But I'd rather dislike if we would define a bunch of gatekeepers who have to approve merge request from contributors. So I'd prefer if we were liberal with giving access to the main gnunet repo, and not create some heavy gatekeeping policies. - Florian """ Error parsing header X-XSS-Protection: 1; mode=block, 1; mode=block: expected semicolon at character position 13. The default protections will be applied. Refused to load the image '' because it violates the following Content Security Policy directive: "default-src 'self'". Note that 'img-src' was not explicitly set, so 'default-src' is used as a fallback. document-register-element.node.js:1296 [Deprecation] document.registerElement is deprecated and will be removed in M73, around March 2019. Please use window.customElements.define instead. See https://www.chromestatus.com/features/4642138092470272 for more details. """ On 4/5/19 9:20 PM, Devan C. dvn wrote: > Hello my fellow GNUnetians, > > As some of you know, I have been pushing for and working on getting us > to CI/CD system based on Gitlab CI. This is pretty much ready to start > using and building out the pipelines for now. > > In a previous thread[0] there was a decision to give Gitlab a try, as > more than just our CI system, and to migrate our repos from Gitolite to > our own Gitlab instance. > There was less agreement regarding the idea to move from MantisBT to > Gitlab's issue tracker, but in the end we decided to try importing all > of the >2k Mantis tickets into Gitlab.[1] > > This email is a status update on the overall progress of moving to > Gitlab. > > Current Gitlab status: > > - Gitlab is running and accessible at `https://gitlab.gnunet.org` > > - Registration is open. There are no guarantees on uptime, or even > data retention (though I don't expect data to disappear). > > - wldhx has kindly offered two "Gitlab Runners" for running CI jobs. > These will be added as shared runners, to be used by any projects on > the instance. This may be changed later to only be shared by projects > under the GNUnet namespace. > > **TODO** > > - Setup email. Used for registration, password resets, > notifcations, and interaction (eg. issue threads). > > - Currently run using containers with docker-compose. Will switch to > using systemd services to with the containers, removing docker-compose. > > - Daily remote backups. Perhaps 'firefly.gnunet.org' could be the backup > site, hmm? > > - Change configured hostname (in Gitlab) to 'gitlab.gnunet.org'. > > - Setup redirect from 'git.gnunet.org' -> 'gitlab.gnunet.org' > > > Current [MantisBT -> Gitlab Issues] status: > > Exporting: > - Mantis can export to CSV and "Excell" XML > - These do not contain comments (bugnotes). It looks like there might > be a possibility to enable them via a configuration option[2]. Not > sure who all has admin access, whom I could coordinate with. Maybe > easier if I can get admin rights? Grothoff, what do you think? > > Importing: > - I have found only a couple of scripts[3,4] for this. They are both out > of date, for old versions of both softwares. I have tried both to no > avail. [4] is the most promising; It's not so old. > I would really appreciate any help working on this. > > I suppose this means that we will continue to use Mantis, and disable > issues in Gitlab for now. Any protests or ideas? > > > Migrating from Gitolite: > > For those whom are not aware, we currently use gitolite for all of the > lovely repos in our collection. We will need to copy
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
wldhx transcribed 4.1K bytes: > > So instead of "hey, signup and we give you access", what about the > > addition of LDAP? > > https://docs.gitlab.com/ee/administration/auth/how_to_configure_ldap_gitlab_ce/ > > Is there additional benefit to LDAP as compared to standard GL ACL [0]? > Note only some roles have "add members" privilege. Probably only to unify additional login accounts we might want for example for email boxes. I have no experience with gitlab ldap. It was just a question. > I had had experience with them being somewhat rigid (you sometimes want > a specific set of privileges which none of the built-in roles covers), > but we should probably be fine. > > >> So here's a problem I see with this as it is right now: > >> I'm a git admin. Before I give people a certain kind of access, be > >> it for one repo only, a range of repos or the group 'gnunet', I > >> have a sort of checklist. Can I digitally verify to some extent that the > >> key sent to me matches the person? Do we have a CAA signature? etc. > >> Now I see already one name as 'Owner' in the gnunet group who, to > >> my knowledge, has never signed anything. Correct me if I'm wrong > >> about ic.rbow. > > ic.rbow has indeed not signed CAA yet. I asked them to now. In my > defense, I added them when gitlab.gnunet.org was a mere experiment :) Okay, then it was just delayed communication. Thanks for clearing that up. > >> We can only trust each other. > >> Since we have this CAA in place, we need more than trust, we need > >> some guidelines when someone is added to which permission level > >> in gitlab. > >> Previously the communication about what happened, which steps > >> were followed and that there is a new committer, were betwee > >> 1 or 2 people involved in administration. Now potentially everyone > >> can do this, which is either bad or good, so at the very least > >> we need to communicate about new rights given. > > +1 for guidelines, +1 for communication, but maybe not that much changes > due to GL ACL. Okay, thanks for your message. > [0]: https://gitlab.com/help/user/permissions.md > > ___ > GNUnet-developers mailing list > GNUnet-developers@gnu.org > https://lists.gnu.org/mailman/listinfo/gnunet-developers signature.asc Description: PGP signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
> So instead of "hey, signup and we give you access", what about the > addition of LDAP? > https://docs.gitlab.com/ee/administration/auth/how_to_configure_ldap_gitlab_ce/ Is there additional benefit to LDAP as compared to standard GL ACL [0]? Note only some roles have "add members" privilege. I had had experience with them being somewhat rigid (you sometimes want a specific set of privileges which none of the built-in roles covers), but we should probably be fine. >> So here's a problem I see with this as it is right now: >> I'm a git admin. Before I give people a certain kind of access, be >> it for one repo only, a range of repos or the group 'gnunet', I >> have a sort of checklist. Can I digitally verify to some extent that the >> key sent to me matches the person? Do we have a CAA signature? etc. >> Now I see already one name as 'Owner' in the gnunet group who, to >> my knowledge, has never signed anything. Correct me if I'm wrong >> about ic.rbow. ic.rbow has indeed not signed CAA yet. I asked them to now. In my defense, I added them when gitlab.gnunet.org was a mere experiment :) >> We can only trust each other. >> Since we have this CAA in place, we need more than trust, we need >> some guidelines when someone is added to which permission level >> in gitlab. >> Previously the communication about what happened, which steps >> were followed and that there is a new committer, were betwee >> 1 or 2 people involved in administration. Now potentially everyone >> can do this, which is either bad or good, so at the very least >> we need to communicate about new rights given. +1 for guidelines, +1 for communication, but maybe not that much changes due to GL ACL. [0]: https://gitlab.com/help/user/permissions.md signature.asc Description: OpenPGP digital signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
On 4/6/19 12:09 PM, n...@n0.is wrote: > How much space for additional harddrives do they have? As a group or > as association we could certainly pay for more disks if possible. 500 GB /home on sam (~70% used), 8 TB /home on firefly (for now empty). sam has no empty bays, firefly has some. But firefly right now doesn't do RAID. My plan is to use the empty bays on firefly for expanding the RAID. And I can probably get the BFH to get the extra disks on firefly. For sam, it'll be more difficult, there we should eventually plan an upgrade, but it'll require physical presence in Munich as well... signature.asc Description: OpenPGP digital signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
n...@n0.is transcribed 8.0K bytes: > Devan C. dvn transcribed 6.5K bytes: > > Hello my fellow GNUnetians, > > > > As some of you know, I have been pushing for and working on getting us > > to CI/CD system based on Gitlab CI. This is pretty much ready to start > > using and building out the pipelines for now. > > > > In a previous thread[0] there was a decision to give Gitlab a try, as > > more than just our CI system, and to migrate our repos from Gitolite to > > our own Gitlab instance. > > There was less agreement regarding the idea to move from MantisBT to > > Gitlab's issue tracker, but in the end we decided to try importing all > > of the >2k Mantis tickets into Gitlab.[1] > > > > This email is a status update on the overall progress of moving to > > Gitlab. > > Cool, and thanks for your work on this :) > > > Current Gitlab status: > > > > - Gitlab is running and accessible at `https://gitlab.gnunet.org` So instead of "hey, signup and we give you access", what about the addition of LDAP? https://docs.gitlab.com/ee/administration/auth/how_to_configure_ldap_gitlab_ce/ > So here's a problem I see with this as it is right now: > I'm a git admin. Before I give people a certain kind of access, be > it for one repo only, a range of repos or the group 'gnunet', I > have a sort of checklist. Can I digitally verify to some extent that the > key sent to me matches the person? Do we have a CAA signature? etc. > Now I see already one name as 'Owner' in the gnunet group who, to > my knowledge, has never signed anything. Correct me if I'm wrong > about ic.rbow. > We can only trust each other. > Since we have this CAA in place, we need more than trust, we need > some guidelines when someone is added to which permission level > in gitlab. > Previously the communication about what happened, which steps > were followed and that there is a new committer, were betwee > 1 or 2 people involved in administration. Now potentially everyone > can do this, which is either bad or good, so at the very least > we need to communicate about new rights given. > > > - Registration is open. There are no guarantees on uptime, or even > > data retention (though I don't expect data to disappear). > > > > - wldhx has kindly offered two "Gitlab Runners" for running CI jobs. > > These will be added as shared runners, to be used by any projects on > > the instance. This may be changed later to only be shared by projects > > under the GNUnet namespace. > > > > **TODO** > > > > - Setup email. Used for registration, password resets, > > notifcations, and interaction (eg. issue threads). > > > > - Currently run using containers with docker-compose. Will switch to > > using systemd services to with the containers, removing docker-compose. > > > > - Daily remote backups. Perhaps 'firefly.gnunet.org' could be the backup > > site, hmm? > > > > - Change configured hostname (in Gitlab) to 'gitlab.gnunet.org'. > > > > - Setup redirect from 'git.gnunet.org' -> 'gitlab.gnunet.org' > > > > > > Current [MantisBT -> Gitlab Issues] status: > > > > Exporting: > > - Mantis can export to CSV and "Excell" XML > > - These do not contain comments (bugnotes). It looks like there might > > be a possibility to enable them via a configuration option[2]. Not > > sure who all has admin access, whom I could coordinate with. Maybe > > easier if I can get admin rights? Grothoff, what do you think? > > > > Importing: > > - I have found only a couple of scripts[3,4] for this. They are both out > > of date, for old versions of both softwares. I have tried both to no > > avail. [4] is the most promising; It's not so old. > > I would really appreciate any help working on this. > > > > I suppose this means that we will continue to use Mantis, and disable > > issues in Gitlab for now. Any protests or ideas? > > > > > > Migrating from Gitolite: > > > > For those whom are not aware, we currently use gitolite for all of the > > lovely repos in our collection. We will need to copy all of the repos to > > Gitlab, as well as duplicate permissions, and setup githooks. > > > > 1. Create namespaces/groups on our Gitlab > > > > 2. Clone repos. This can be done via the web interface "Import" option > > when creating a new repo, or the new remote can be added, and then > > pushed. The little script found here can help with getting all the > > repos from Gitolite[5] > > > > 3. Setup redirects. eg. https://gnunet.org/git -> > > https://gitlab.gnunet.org > > > > 4. Manually replicate permissions. Will need a Gitolite admin's help > > on this. > > > > 5. Setup githooks. We have quite a few githooks setup, so we will > > need to recreate those. > > > > After all of that is done, I think we should be ready to switch over > > to Gitlab for at least the git management and CI/CD. > > > > > > That brings us to the final update: The CI System... > > > > - We have a couple of small runners (thanks wldhx). >
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
Devan C. dvn transcribed 6.5K bytes: > Hello my fellow GNUnetians, > > As some of you know, I have been pushing for and working on getting us > to CI/CD system based on Gitlab CI. This is pretty much ready to start > using and building out the pipelines for now. > > In a previous thread[0] there was a decision to give Gitlab a try, as > more than just our CI system, and to migrate our repos from Gitolite to > our own Gitlab instance. > There was less agreement regarding the idea to move from MantisBT to > Gitlab's issue tracker, but in the end we decided to try importing all > of the >2k Mantis tickets into Gitlab.[1] > > This email is a status update on the overall progress of moving to > Gitlab. Cool, and thanks for your work on this :) > Current Gitlab status: > > - Gitlab is running and accessible at `https://gitlab.gnunet.org` So here's a problem I see with this as it is right now: I'm a git admin. Before I give people a certain kind of access, be it for one repo only, a range of repos or the group 'gnunet', I have a sort of checklist. Can I digitally verify to some extent that the key sent to me matches the person? Do we have a CAA signature? etc. Now I see already one name as 'Owner' in the gnunet group who, to my knowledge, has never signed anything. Correct me if I'm wrong about ic.rbow. We can only trust each other. Since we have this CAA in place, we need more than trust, we need some guidelines when someone is added to which permission level in gitlab. Previously the communication about what happened, which steps were followed and that there is a new committer, were betwee 1 or 2 people involved in administration. Now potentially everyone can do this, which is either bad or good, so at the very least we need to communicate about new rights given. > - Registration is open. There are no guarantees on uptime, or even > data retention (though I don't expect data to disappear). > > - wldhx has kindly offered two "Gitlab Runners" for running CI jobs. > These will be added as shared runners, to be used by any projects on > the instance. This may be changed later to only be shared by projects > under the GNUnet namespace. > > **TODO** > > - Setup email. Used for registration, password resets, > notifcations, and interaction (eg. issue threads). > > - Currently run using containers with docker-compose. Will switch to > using systemd services to with the containers, removing docker-compose. > > - Daily remote backups. Perhaps 'firefly.gnunet.org' could be the backup > site, hmm? > > - Change configured hostname (in Gitlab) to 'gitlab.gnunet.org'. > > - Setup redirect from 'git.gnunet.org' -> 'gitlab.gnunet.org' > > > Current [MantisBT -> Gitlab Issues] status: > > Exporting: > - Mantis can export to CSV and "Excell" XML > - These do not contain comments (bugnotes). It looks like there might > be a possibility to enable them via a configuration option[2]. Not > sure who all has admin access, whom I could coordinate with. Maybe > easier if I can get admin rights? Grothoff, what do you think? > > Importing: > - I have found only a couple of scripts[3,4] for this. They are both out > of date, for old versions of both softwares. I have tried both to no > avail. [4] is the most promising; It's not so old. > I would really appreciate any help working on this. > > I suppose this means that we will continue to use Mantis, and disable > issues in Gitlab for now. Any protests or ideas? > > > Migrating from Gitolite: > > For those whom are not aware, we currently use gitolite for all of the > lovely repos in our collection. We will need to copy all of the repos to > Gitlab, as well as duplicate permissions, and setup githooks. > > 1. Create namespaces/groups on our Gitlab > > 2. Clone repos. This can be done via the web interface "Import" option > when creating a new repo, or the new remote can be added, and then > pushed. The little script found here can help with getting all the > repos from Gitolite[5] > > 3. Setup redirects. eg. https://gnunet.org/git -> > https://gitlab.gnunet.org > > 4. Manually replicate permissions. Will need a Gitolite admin's help > on this. > > 5. Setup githooks. We have quite a few githooks setup, so we will > need to recreate those. > > After all of that is done, I think we should be ready to switch over > to Gitlab for at least the git management and CI/CD. > > > That brings us to the final update: The CI System... > > - We have a couple of small runners (thanks wldhx). > > - We have some very basic '.gitlab-ci.yml'[6] files, defining jobs. > - I will begin expanding these in the coming weeks. > > **TODO** > > - As we build out a matrix of pipelines, we will need more resources. > 'firefly.gnunet.org' is a logical option. In the past I've seen it > utilized heavily by experiments. As long as we are okay with dedicating > some CPU and RAM to runners, then I will begin setting them up. >
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
Christian Grothoff transcribed 4.5K bytes: > On 4/5/19 9:20 PM, Devan C. dvn wrote: > > - wldhx has kindly offered two "Gitlab Runners" for running CI jobs. > > These will be added as shared runners, to be used by any projects on > > the instance. This may be changed later to only be shared by projects > > under the GNUnet namespace. > > Eh, what is required here? It's not like we don't have enough CPU or RAM > on gnunet.org itself, right? > > > - Daily remote backups. Perhaps 'firefly.gnunet.org' could be the backup > > site, hmm? > > I plan to re-install firefly "soon", after that it should ideally become > the primary site in the future. > > > Exporting: > > - Mantis can export to CSV and "Excell" XML > > - These do not contain comments (bugnotes). It looks like there might > > be a possibility to enable them via a configuration option[2]. Not > > sure who all has admin access, whom I could coordinate with. Maybe > > easier if I can get admin rights? Grothoff, what do you think? > > I've checked the configuration option, and it can only give you the # > bug notes, not the bug notes as far as I can tell. I think you will > need to do a *proper* DB export->import. Also, the CSV will hardly give > you the file attachments either. I think losing the file-attachments would be okay if that means we get the migration done for the rest. > > I suppose this means that we will continue to use Mantis, and disable > > issues in Gitlab for now. Any protests or ideas? > > Neither ideas nor protest from me at this time ;-). > > > 4. Manually replicate permissions. Will need a Gitolite admin's help > > on this. > > dvn: you're now gitolite admin, please help yourself ;-). Unless you require general orientation what all of these permissions and special permissions in gitolite grammar mean. Feel free to ask if you need to. > > - As we build out a matrix of pipelines, we will need more resources. > > 'firefly.gnunet.org' is a logical option. In the past I've seen it > > utilized heavily by experiments. As long as we are okay with dedicating > > some CPU and RAM to runners, then I will begin setting them up. > > How much crazy resources are you expecting to need here? You have ~20 > idle cores and 40 GB free RAM on gnunet.org --- and that's with the > existing buildbot CI running. If gitlab is _that_ resource hungry, maybe > it's not a viable option? > > > - Setup Gitlab Container Registry [7] for storing our CI artifacts. > > That's more worrisome, we're short-ish on disk space on _both_ systems. > So please be careful there! How much space for additional harddrives do they have? As a group or as association we could certainly pay for more disks if possible. > > Happy hacking! > > Christian > > ___ > GNUnet-developers mailing list > GNUnet-developers@gnu.org > https://lists.gnu.org/mailman/listinfo/gnunet-developers signature.asc Description: PGP signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
Thanks for the writeup. My comments below. > On 5. Apr 2019, at 21:20, Devan C. dvn wrote: > > Signed PGP part > Hello my fellow GNUnetians, > > As some of you know, I have been pushing for and working on getting us > to CI/CD system based on Gitlab CI. This is pretty much ready to start > using and building out the pipelines for now. > > In a previous thread[0] there was a decision to give Gitlab a try, as > more than just our CI system, and to migrate our repos from Gitolite to > our own Gitlab instance. > There was less agreement regarding the idea to move from MantisBT to > Gitlab's issue tracker, but in the end we decided to try importing all > of the >2k Mantis tickets into Gitlab.[1] > > This email is a status update on the overall progress of moving to > Gitlab. > > Current Gitlab status: > > - Gitlab is running and accessible at `https://gitlab.gnunet.org` > > - Registration is open. There are no guarantees on uptime, or even > data retention (though I don't expect data to disappear). > > - wldhx has kindly offered two "Gitlab Runners" for running CI jobs. > These will be added as shared runners, to be used by any projects on > the instance. This may be changed later to only be shared by projects > under the GNUnet namespace. > > **TODO** > > - Setup email. Used for registration, password resets, > notifcations, and interaction (eg. issue threads). > > - Currently run using containers with docker-compose. Will switch to > using systemd services to with the containers, removing docker-compose. > > - Daily remote backups. Perhaps 'firefly.gnunet.org' could be the backup > site, hmm? > > - Change configured hostname (in Gitlab) to 'gitlab.gnunet.org'. > > - Setup redirect from 'git.gnunet.org' -> 'gitlab.gnunet.org' > > > Current [MantisBT -> Gitlab Issues] status: > > Exporting: > - Mantis can export to CSV and "Excell" XML > - These do not contain comments (bugnotes). It looks like there might >be a possibility to enable them via a configuration option[2]. Not >sure who all has admin access, whom I could coordinate with. Maybe >easier if I can get admin rights? Grothoff, what do you think? > > Importing: > - I have found only a couple of scripts[3,4] for this. They are both out > of date, for old versions of both softwares. I have tried both to no > avail. [4] is the most promising; It's not so old. > I would really appreciate any help working on this. > > I suppose this means that we will continue to use Mantis, and disable > issues in Gitlab for now. Any protests or ideas? That would be very sad. And fwiw, I think this is because Mantis does not offer proper export/APIs... We should migrate to gitlab issues if we decide for it. Even if that means a hard break with legacy Mantis issues. > > > Migrating from Gitolite: > > For those whom are not aware, we currently use gitolite for all of the > lovely repos in our collection. We will need to copy all of the repos to > Gitlab, as well as duplicate permissions, and setup githooks. > > 1. Create namespaces/groups on our Gitlab > > 2. Clone repos. This can be done via the web interface "Import" option > when creating a new repo, or the new remote can be added, and then > pushed. The little script found here can help with getting all the > repos from Gitolite[5] > > 3. Setup redirects. eg. https://gnunet.org/git -> > https://gitlab.gnunet.org > > 4. Manually replicate permissions. Will need a Gitolite admin's help > on this. Can't we just have uses use their own namespaces for development? I mean, the power of gitlab is that users work in their own forks and create pull requests. Then, the "main" project repo does some testing/ci magic and if everything passes, it is (automatically) merged. No need to give everybody access manually! > > 5. Setup githooks. We have quite a few githooks setup, so we will > need to recreate those. > > After all of that is done, I think we should be ready to switch over > to Gitlab for at least the git management and CI/CD. > > > That brings us to the final update: The CI System... > > - We have a couple of small runners (thanks wldhx). > > - We have some very basic '.gitlab-ci.yml'[6] files, defining jobs. > - I will begin expanding these in the coming weeks. > > **TODO** > > - As we build out a matrix of pipelines, we will need more resources. > 'firefly.gnunet.org' is a logical option. In the past I've seen it > utilized heavily by experiments. As long as we are okay with dedicating > some CPU and RAM to runners, then I will begin setting them up. > > - Setup Gitlab Container Registry [7] for storing our CI artifacts. > > - Expand our '.gitlab-ci.yml' files to include e2e tests, builds for > multiple architecures, and continuous delivery of packages for various > package managers. > > > Wow, so that's a lot of text. A lot of people have been asking me about > the status of Gitlab, and if and how they can help with CI. I hope this >
Re: [GNUnet-developers] Discussion, and Help Wanted: Moving to Gitlab for Git, CI, and Issues
On 4/5/19 9:20 PM, Devan C. dvn wrote: > - wldhx has kindly offered two "Gitlab Runners" for running CI jobs. > These will be added as shared runners, to be used by any projects on > the instance. This may be changed later to only be shared by projects > under the GNUnet namespace. Eh, what is required here? It's not like we don't have enough CPU or RAM on gnunet.org itself, right? > - Daily remote backups. Perhaps 'firefly.gnunet.org' could be the backup > site, hmm? I plan to re-install firefly "soon", after that it should ideally become the primary site in the future. > Exporting: > - Mantis can export to CSV and "Excell" XML > - These do not contain comments (bugnotes). It looks like there might > be a possibility to enable them via a configuration option[2]. Not > sure who all has admin access, whom I could coordinate with. Maybe > easier if I can get admin rights? Grothoff, what do you think? I've checked the configuration option, and it can only give you the # bug notes, not the bug notes as far as I can tell. I think you will need to do a *proper* DB export->import. Also, the CSV will hardly give you the file attachments either. > I suppose this means that we will continue to use Mantis, and disable > issues in Gitlab for now. Any protests or ideas? Neither ideas nor protest from me at this time ;-). > 4. Manually replicate permissions. Will need a Gitolite admin's help > on this. dvn: you're now gitolite admin, please help yourself ;-). > - As we build out a matrix of pipelines, we will need more resources. > 'firefly.gnunet.org' is a logical option. In the past I've seen it > utilized heavily by experiments. As long as we are okay with dedicating > some CPU and RAM to runners, then I will begin setting them up. How much crazy resources are you expecting to need here? You have ~20 idle cores and 40 GB free RAM on gnunet.org --- and that's with the existing buildbot CI running. If gitlab is _that_ resource hungry, maybe it's not a viable option? > - Setup Gitlab Container Registry [7] for storing our CI artifacts. That's more worrisome, we're short-ish on disk space on _both_ systems. So please be careful there! Happy hacking! Christian signature.asc Description: OpenPGP digital signature ___ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
