Re: modular protobuf issue (Dec. 6, 2019) recap
On Sat, Dec 7, 2019 at 2:57 PM Zbigniew Jędrzejewski-Szmek wrote: > > On Fri, Dec 06, 2019 at 08:18:21PM -0500, Langdon White wrote: > > ## How to determine if you have an issue and how to fix it: > > > > run: ```sudo dnf list --installed *protobuf*``` > > if you get a result that looks like > > ```protobuf.x86_64 3.6.1-6.module_f31+6793+1c93c38``` > > > > you have encountered the problem. so please: > > > > run: ```sudo dnf module disable eclipse``` > > run: ```sudo dnf downgrade protobuf``` > > > > Any updates, as of a few hours ago, should be fine. > > > > ## What happened: > > First and foremost, this issue appears to be caused by Modularity but, in > > fact, is just an example of a policy not being followed. > > This is correct, but at the same time imo very short-sighted. > Look at it this way: setting the eclipse module as default was discussed > in the FESCo issue tracker and in IRC meetings for about two months. > I think that this particular stream had undergone scrutiny that is > _way above normal_. The requests for additional information in the > FESCo ticket were overruled. Despite that when it gets enabled things > go south and require manual workarounds. It seems that introspectability > of modules is not good and even the people who designed Modularity > can't do the required checks. > > Now let's imagine that we don't have just a handful of streams, most > of them very trivial, but a few hundred, including some that build > complicated dependency trees, with actual multiple versions of dependencies. > Nobody will have the time or energy to do introspection, so those kinds > of issues will go undiscovered until reported by users. How often would > such issues occur, e.g. a few hundred times per year? > > For this it doesn't really matter if the modules are default or not: > either enabled as a default or explicitly by the user, modules *will* > override other packages they shouldn't, sometimes by mistake, > sometimes on purpose as a result of a disagreement between maintainers. > With modularity, any module owner has effectively *higher* privileges > than owners of packages, because modules override non-modular builds. (snip) > Why do I think this is more of a problem with modules? Right now we > have an elaborate and long-established system with maintainers, > co-maintainers, proven packagers, and mass changes, i.e. some formal > and some customary rules as to what can be done to packages of other > maintainers and who is "at fault" with any given bug. Modularity > throws this system out the window and replaces it with ... nothing. I agree - Modularity (at least, in its current incarnation) has made it way too easy to make mistakes (whether on purpose or by accident), with no good way to undo the damage once they have been pushed to end users. Fabio > Zbyszek > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: modular protobuf issue (Dec. 6, 2019) recap
Am 07.12.19 um 12:17 schrieb Miro Hrončok: And disallow all the current default modular streams. Ship defaults as traditional RPMs. Keep modularity for alternate versions. +1 With the current tooling modularity must be optional as it requires manual steps to fix packaging problems. With default modular streams modularity is just not optional. I think modularity might be a way in *exceptional cases* (alternate versions) but with the current tooling I'm not even sure with that. IMHO modules mean that we create sub-distributions thus splitting our package base (and more importantly out packager community!). This is not only a technical problem but also a social one. Felix ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: modular protobuf issue (Dec. 6, 2019) recap
On Sat, 2019-12-07 at 10:42 +0100, Kevin Kofler wrote: > Adam Williamson wrote: > > I really think we should recommend 'dnf distro-sync', assuming it > > actually does the job, as that should return *all* installed packages > > from the newly-disabled modules to the non-modular builds, not just > > protobuf. > > Unfortunately, you also need to "dnf module disable *" explicitly first. Yes, obviously - but I meant we should recommend 'distro-sync' as the *second* step, rather than 'downgrade protobuf'. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: modular protobuf issue (Dec. 6, 2019) recap
- Original Message - > From: "Zbigniew Jędrzejewski-Szmek" > To: "Development discussions related to Fedora" > > Sent: Saturday, December 7, 2019 2:55:30 PM > Subject: Re: modular protobuf issue (Dec. 6, 2019) recap > > On Fri, Dec 06, 2019 at 08:18:21PM -0500, Langdon White wrote: > > ## How to determine if you have an issue and how to fix it: > > > > run: ```sudo dnf list --installed *protobuf*``` > > if you get a result that looks like > > ```protobuf.x86_64 3.6.1-6.module_f31+6793+1c93c38``` > > > > you have encountered the problem. so please: > > > > run: ```sudo dnf module disable eclipse``` > > run: ```sudo dnf downgrade protobuf``` > > > > Any updates, as of a few hours ago, should be fine. > > > > ## What happened: > > First and foremost, this issue appears to be caused by Modularity but, in > > fact, is just an example of a policy not being followed. > > This is correct, but at the same time imo very short-sighted. > Look at it this way: setting the eclipse module as default was discussed > in the FESCo issue tracker and in IRC meetings for about two months. > I think that this particular stream had undergone scrutiny that is > _way above normal_. The requests for additional information in the > FESCo ticket were overruled. Despite that when it gets enabled things > go south and require manual workarounds. It seems that introspectability > of modules is not good and even the people who designed Modularity > can't do the required checks. > > Now let's imagine that we don't have just a handful of streams, most > of them very trivial, but a few hundred, including some that build > complicated dependency trees, with actual multiple versions of dependencies. > Nobody will have the time or energy to do introspection, so those kinds > of issues will go undiscovered until reported by users. How often would > such issues occur, e.g. a few hundred times per year? > > For this it doesn't really matter if the modules are default or not: > either enabled as a default or explicitly by the user, modules *will* > override other packages they shouldn't, sometimes by mistake, > sometimes on purpose as a result of a disagreement between maintainers. > With modularity, any module owner has effectively *higher* privileges > than owners of packages, because modules override non-modular builds. > > Why do I think this is more of a problem with modules? Right now we > have an elaborate and long-established system with maintainers, > co-maintainers, proven packagers, and mass changes, i.e. some formal > and some customary rules as to what can be done to packages of other > maintainers and who is "at fault" with any given bug. Modularity > throws this system out the window and replaces it with ... nothing. > > Zbyszek > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Exactly. It might be a policy issue, but I haven't seen any checks in place to ensure that the policy is being followed. The only way to figure out a policy violation is being reported by someone if at all. And if as a module maintainer want to override non-modular rpm's by accident or maliciousness on stable systems, now I get that capability with modularity. -- Regards, Charalampos Stratakis Software Engineer Python Maintenance Team, Red Hat ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: modular protobuf issue (Dec. 6, 2019) recap
On Fri, Dec 06, 2019 at 08:18:21PM -0500, Langdon White wrote: > ## How to determine if you have an issue and how to fix it: > > run: ```sudo dnf list --installed *protobuf*``` > if you get a result that looks like > ```protobuf.x86_64 3.6.1-6.module_f31+6793+1c93c38``` > > you have encountered the problem. so please: > > run: ```sudo dnf module disable eclipse``` > run: ```sudo dnf downgrade protobuf``` > > Any updates, as of a few hours ago, should be fine. > > ## What happened: > First and foremost, this issue appears to be caused by Modularity but, in > fact, is just an example of a policy not being followed. This is correct, but at the same time imo very short-sighted. Look at it this way: setting the eclipse module as default was discussed in the FESCo issue tracker and in IRC meetings for about two months. I think that this particular stream had undergone scrutiny that is _way above normal_. The requests for additional information in the FESCo ticket were overruled. Despite that when it gets enabled things go south and require manual workarounds. It seems that introspectability of modules is not good and even the people who designed Modularity can't do the required checks. Now let's imagine that we don't have just a handful of streams, most of them very trivial, but a few hundred, including some that build complicated dependency trees, with actual multiple versions of dependencies. Nobody will have the time or energy to do introspection, so those kinds of issues will go undiscovered until reported by users. How often would such issues occur, e.g. a few hundred times per year? For this it doesn't really matter if the modules are default or not: either enabled as a default or explicitly by the user, modules *will* override other packages they shouldn't, sometimes by mistake, sometimes on purpose as a result of a disagreement between maintainers. With modularity, any module owner has effectively *higher* privileges than owners of packages, because modules override non-modular builds. Why do I think this is more of a problem with modules? Right now we have an elaborate and long-established system with maintainers, co-maintainers, proven packagers, and mass changes, i.e. some formal and some customary rules as to what can be done to packages of other maintainers and who is "at fault" with any given bug. Modularity throws this system out the window and replaces it with ... nothing. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: modular protobuf issue (Dec. 6, 2019) recap
Miro Hrončok wrote: > On 07. 12. 19 10:44, Kevin Kofler wrote: >> Langdon White wrote: >>> ## What we can do going forward: >>> * Increase the awareness of the policies for Fedora Modules >>> * Investigate an "early warning system" that would indicate to packagers >>> (modular and RPM) when they might be violating this policy >>> * Request dnf notify the user when they are enabling a superseding rpm >>> from a default stream module >>> * Request dnf provide an indication of what module is providing a >>> particular rpm (e.g. `dnf provides protobuf` not just indicate repo >>> origin but also module name and stream) >> >> * Stop allowing default module streams, which are the main reason this >> was >>allowed to happen, and which are just asking for this kind of >>conflicts. > > And disallow all the current default modular streams. Ship defaults as > traditional RPMs. Keep modularity for alternate versions. +1, indeed. (I would consider that part of "stop allowing", but thanks for the clarification.) Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: modular protobuf issue (Dec. 6, 2019) recap
On 07. 12. 19 2:18, Langdon White wrote: ## How to determine if you have an issue and how to fix it: run: ```sudo dnf list --installed *protobuf*``` if you get a result that looks like ```protobuf.x86_64 3.6.1-6.module_f31+6793+1c93c38``` you have encountered the problem. so please: run: ```sudo dnf module disable eclipse``` run: ```sudo dnf downgrade protobuf``` Since we cannot expect our users to read this list, I have opened: https://bugzilla.redhat.com/show_bug.cgi?id=1780827 This needs to happen implicitly. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: modular protobuf issue (Dec. 6, 2019) recap
On 07. 12. 19 10:44, Kevin Kofler wrote: Langdon White wrote: ## What we can do going forward: * Increase the awareness of the policies for Fedora Modules * Investigate an "early warning system" that would indicate to packagers (modular and RPM) when they might be violating this policy * Request dnf notify the user when they are enabling a superseding rpm from a default stream module * Request dnf provide an indication of what module is providing a particular rpm (e.g. `dnf provides protobuf` not just indicate repo origin but also module name and stream) * Stop allowing default module streams, which are the main reason this was allowed to happen, and which are just asking for this kind of conflicts. And disallow all the current default modular streams. Ship defaults as traditional RPMs. Keep modularity for alternate versions. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: modular protobuf issue (Dec. 6, 2019) recap
Adam Williamson wrote: > I really think we should recommend 'dnf distro-sync', assuming it > actually does the job, as that should return *all* installed packages > from the newly-disabled modules to the non-modular builds, not just > protobuf. Unfortunately, you also need to "dnf module disable *" explicitly first. Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: modular protobuf issue (Dec. 6, 2019) recap
Langdon White wrote: > ## What we can do going forward: > * Increase the awareness of the policies for Fedora Modules > * Investigate an "early warning system" that would indicate to packagers > (modular and RPM) when they might be violating this policy > * Request dnf notify the user when they are enabling a superseding rpm > from a default stream module > * Request dnf provide an indication of what module is providing a > particular rpm (e.g. `dnf provides protobuf` not just indicate repo origin > but also module name and stream) * Stop allowing default module streams, which are the main reason this was allowed to happen, and which are just asking for this kind of conflicts. Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: modular protobuf issue (Dec. 6, 2019) recap
On Fri, Dec 6, 2019 at 8:19 PM Langdon White wrote: > ## What happened: > First and foremost, this issue appears to be caused by Modularity but, in > fact, is just an example of a policy not being followed. When a module wishes > to declare a "default stream"[1] it *must* not override a traditional RPM > without express permission from FESCo. The policy is documented here[2]. > > Unfortunately, the recent change to enable an Eclipse Module Stream as a > default stream did not follow this policy and provided a different protobuf > RPM with different functionality. > > ## What we can do going forward: > * Increase the awareness of the policies for Fedora Modules > * Investigate an "early warning system" that would indicate to packagers > (modular and RPM) when they might be violating this policy Hi Langdon, Thanks for the summary of the issue. Igor wrote in the other thread that, as the maintainer of protobuf, he had no idea this was happening. In addition to FESCo approval, is it worth stating explicitly somewhere in the policy that overriding a traditional RPM should require the consent of the maintainer(s) of that RPM? (I realize that in this particular case, Igor is on FESCo, and so had the policy been followed more carefully, he would have been aware anyway). Ben Rosser ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: modular protobuf issue (Dec. 6, 2019) recap
On 07. 12. 19 2:18, Langdon White wrote: First and foremost, this issue appears to be caused by Modularity but, in fact, is just an example of a policy not being followed. When a module wishes to declare a "default stream"[1] it *must* not override a traditional RPM without express permission from FESCo. The policy is documented here[2]. To clarify: The default modular stream of eclipse was approved by FESCo (and it was known that it would override traditional packages), but it was approved based on incomplete information. The need for the default stream however, was caused by other default modular streams violating this policy. Is this the right time to remove such default modular streams? -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: modular protobuf issue (Dec. 6, 2019) recap
On Fri, 2019-12-06 at 20:18 -0500, Langdon White wrote: > ## How to determine if you have an issue and how to fix it: > > run: ```sudo dnf list --installed *protobuf*``` > if you get a result that looks like > ```protobuf.x86_64 3.6.1-6.module_f31+6793+1c93c38``` > > you have encountered the problem. so please: > > run: ```sudo dnf module disable eclipse``` > run: ```sudo dnf downgrade protobuf``` > > Any updates, as of a few hours ago, should be fine. I really think we should recommend 'dnf distro-sync', assuming it actually does the job, as that should return *all* installed packages from the newly-disabled modules to the non-modular builds, not just protobuf. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
modular protobuf issue (Dec. 6, 2019) recap
## How to determine if you have an issue and how to fix it: run: ```sudo dnf list --installed *protobuf*``` if you get a result that looks like ```protobuf.x86_64 3.6.1-6.module_f31+6793+1c93c38``` you have encountered the problem. so please: run: ```sudo dnf module disable eclipse``` run: ```sudo dnf downgrade protobuf``` Any updates, as of a few hours ago, should be fine. ## What happened: First and foremost, this issue appears to be caused by Modularity but, in fact, is just an example of a policy not being followed. When a module wishes to declare a "default stream"[1] it *must* not override a traditional RPM without express permission from FESCo. The policy is documented here[2]. Unfortunately, the recent change to enable an Eclipse Module Stream as a default stream did not follow this policy and provided a different protobuf RPM with different functionality. ## What we can do going forward: * Increase the awareness of the policies for Fedora Modules * Investigate an "early warning system" that would indicate to packagers (modular and RPM) when they might be violating this policy * Request dnf notify the user when they are enabling a superseding rpm from a default stream module * Request dnf provide an indication of what module is providing a particular rpm (e.g. `dnf provides protobuf` not just indicate repo origin but also module name and stream) Langdon [1]: if you are unfamiliar, this is the functionality that allows a module stream to be used transparently by end users without them needing to explicitly install a module. [2]: https://docs.fedoraproject.org/en-US/modularity/making-modules/managing-defaults/#_setting_or_changing_a_default ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
