Re: Dependencies on Bodhi Updates
> > Having this implemented manually would be great. In the future I'd > > like to replace it with automatic process managed by AutoQA. AutoQA > > would say Bodhi "this update can be only pushed together with this > > other update, because the first one depends on the second one". The > > maintainer wouldn't then be forced to create mega-updates for > > dependent packages. > > I'm not sure we necessarily want AutoQA to be _driving_ processes > like > this, do we? Properly, it should be _verifying_ that updates are > dependency-complete, not performing the task of making them > dependency-complete. Doesn't it work better to have the > dependencies-in-updates stuff handled at the level of the existing > tools > by which developers work with updates - i.e. Bodhi? Has AutoQA really > been designed to the requirements of acting as a traffic cop for > Bodhi? I'm not sure I follow you on this. I was talking about verification. Let's have updates A and B in Bodhi, A depends on B. AutoQA will say "pushing A+B is fine", "pushing A alone is not fine", "pushing B alone is fine". That means the developer will not be required to manually specify "my update A depends on update B", because we wouldn't push A alone anyway. But I don't think this is happening too soon so this discussion is really academical. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Dependencies on Bodhi Updates
On Mon, 2012-04-02 at 05:39 -0400, Kamil Paral wrote: > > So I really see two options for improving these situations: > > 1) https://fedorahosted.org/bodhi/ticket/663 I opened this ticket two > > months ago (to silence). The idea would be to add the ability for > > bodhi > > updates to mark other updates as a dependency, so that in the example > > above, Firefox could have been marked as ready for stable, but not > > pushed until the nss update was also marked as ready for stable. This > > to > > me seems like the best long-term solution. I'd also like to mention > > that > > Ubuntu's Launchpad system has this capability. > > Having this implemented manually would be great. In the future I'd > like to replace it with automatic process managed by AutoQA. AutoQA > would say Bodhi "this update can be only pushed together with this > other update, because the first one depends on the second one". The > maintainer wouldn't then be forced to create mega-updates for > dependent packages. I'm not sure we necessarily want AutoQA to be _driving_ processes like this, do we? Properly, it should be _verifying_ that updates are dependency-complete, not performing the task of making them dependency-complete. Doesn't it work better to have the dependencies-in-updates stuff handled at the level of the existing tools by which developers work with updates - i.e. Bodhi? Has AutoQA really been designed to the requirements of acting as a traffic cop for Bodhi? -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Dependencies on Bodhi Updates
On 03/26/2012 09:53 PM, Stephen Gallagher wrote: As requested during the FESCo meeting, I am going to try to summarize some of the issues inherent in the way that Bodhi updates currently work. First, I'll try to explain the goals and constraints: 1) The stable 'fedora-updates' yum repository should NEVER exist in a state where any package has dependency issues. In other words, it should never be possible for an update to be pushed to stable that breaks cleanly updating any other package. 2) Updates must be possible and (ideally) timely. This is probably self-evident. 3) Packages pushed to the stable 'fedora-updates' yum repository should (ideally) not introduce regressions in packages that depend on them. 4) New features in "superpackages" such as Firefox, GNOME or FreeIPA that have many and varied dependencies may require new features in packages they depend on in order to enhance or fix the superpackage. In the trivial example, a package (let's say libtalloc) needs to make an update to fix a bug. This package requires nothing new from its dependencies and is a self-contained fix. For this example, it is simple to just build libtalloc in koji and then create a Bodhi update and pass it through "updates-testing", get karma and *poof* off to "fedora-updates". Now let's extend the example. Suppose that we have another package libtevent that has libtalloc as a dependency. Libtevent's maintainer wants to add a new feature to libtevent, but the patch from upstream depends on the bug in libtalloc having been fixed in order for the new feature to work properly. In this situation, the maintainer of libtevent would build libtevent with an explicit Requires: libtalloc>= in the specfile (possibly pulling libtalloc into the BuildRoot overrides if necessary) and then test it locally to see that it works. So now we have our first updates dependency issue. If we submit libtevent as its own update, it is possible that it will achieve its karma requirement before libtalloc does. It would then be pushed to the "fedora-updates" repository and then introduce a dependency issue in the stable repository (because users trying to update libtevent would be unable to update libtalloc without enabling the updates-testing repository). The current recommended approach is to bundle the two updates into a single one carrying multiple packages. The first problem with this is that you must have commit privilege on all packages that you are bundling into an update. If you do not, then you need to track down a provenpackager to do it for you. Now let's make the problem even more fun. Consider that the update to libtevent might be coming in because it is necessary for a new feature in libldb, which is in turn providing new functionality necessary for SSSD. So now we have four packages all sitting in the same update. The problem with this is that the tendency will be to only test the most user-visible package(s) in the set. In this particular case, that might be SSSD. So people would likely test SSSD and, if nothing went wrong, consider the entire update stable. But wait! SSSD isn't the only package that depends on libldb, libtevent and libtalloc. So too does the samba package. Suppose that the bugfix in libtalloc, after resolving the original issue, results in exposing another more serious bug in samba? Now we need to pull a samba update into this same update series. A contrived example, you say? That would never happen, bugfixes aren't likely to do that. Well, for one example: https://admin.fedoraproject.org/updates/FEDORA-2011-11845 In this particular example, we knew up-front that it was going to necessitate a rebuild of several dependent packages and we coordinated a single release to address them. So in this case, the proper approach was to bundle them together in a single update. This worked because we specifically knew that the libtevent change was going to break other packages. But what about when we don't know that? Let's take another example: https://admin.fedoraproject.org/updates/FEDORA-2011-17399 In this case, there was a security bug reported against Firefox. Such things are serious, and acted on quickly. However, the bug was actually fixed in the nss package, and Firefox, Xulrunner and friends were rebuilt against that nss package. The problem was this: the fix made to the nss package introduced regressions in every other package that depended on it. However, because the default install of Firefox contained no issues, it rapidly received the necessary karma points and the whole update was pushed to stable. It then broke nearly every application in Fedora that relied on cryptography. The problem here was sociological, not technological. The only package that received testing was Firefox. It's hard to say without evidence whether the problem would have been averted by having nss go through its own update, but I strongly suspect that what we would have seen was greater testing on actual nss features for that spe
Re: Dependencies on Bodhi Updates
> So I really see two options for improving these situations: > 1) https://fedorahosted.org/bodhi/ticket/663 I opened this ticket two > months ago (to silence). The idea would be to add the ability for > bodhi > updates to mark other updates as a dependency, so that in the example > above, Firefox could have been marked as ready for stable, but not > pushed until the nss update was also marked as ready for stable. This > to > me seems like the best long-term solution. I'd also like to mention > that > Ubuntu's Launchpad system has this capability. Having this implemented manually would be great. In the future I'd like to replace it with automatic process managed by AutoQA. AutoQA would say Bodhi "this update can be only pushed together with this other update, because the first one depends on the second one". The maintainer wouldn't then be forced to create mega-updates for dependent packages. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Dependencies on Bodhi Updates
Adam Williamson wrote: > I think we'd need to make the second more optional than you suggest, > though. For instance, when the desktop team pushes a 'GNOME 3.4' update > with 30 packages in it, they really want that update to be tested as a > whole - broadly they just want people to install all the updates, boot > into GNOME, and make sure stuff mostly works. They probably don't want > the entire update blocked if there's a typo in the Help file for one of > the games, or something. Indeed. If I can't push my KDE SC update group because kfloppy doesn't have karma, I'm going to yell so loud that you'll be able to hear it even in America or Australia (*)! ;-) Kevin Kofler (*) which is NOT anywhere near Austria (where I live), for the geography- challenged among the readers of the mailing list ;-) -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Dependencies on Bodhi Updates
On Mon, 2012-03-26 at 15:53 -0400, Stephen Gallagher wrote: > So I really see two options for improving these situations: > 1) https://fedorahosted.org/bodhi/ticket/663 I opened this ticket two > months ago (to silence). The idea would be to add the ability for bodhi > updates to mark other updates as a dependency, so that in the example > above, Firefox could have been marked as ready for stable, but not > pushed until the nss update was also marked as ready for stable. This to > me seems like the best long-term solution. I'd also like to mention that > Ubuntu's Launchpad system has this capability. > 2) We could continue on the "single update for multiple packages" > approach, but revamp the karma system so that each SRPM gets its own > karma, rather than the update as a whole. Then, the whole update would > not be pushed via autokarma until all of the dependent packages had > sufficient karma (or the owner of the update could push them after the > stable wait period, of course). I think either of these would be an improvement. Both are likely Bodhi 2.0-dependent, though. As I understand the Bodhi 2 design, it would certainly be capable of at least the second (as it's intended to make feedback a much more customizable and granular thing). I think we'd need to make the second more optional than you suggest, though. For instance, when the desktop team pushes a 'GNOME 3.4' update with 30 packages in it, they really want that update to be tested as a whole - broadly they just want people to install all the updates, boot into GNOME, and make sure stuff mostly works. They probably don't want the entire update blocked if there's a typo in the Help file for one of the games, or something. So I think there are cases where an update owner would not want to require the threshold karma on every single sub-package in an update. With that caveat, though, I do like the ideas. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Dependencies on Bodhi Updates
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 26.03.2012 21:53, Stephen Gallagher wrote: > > So now we have our first updates dependency issue. If we submit > libtevent as its own update, it is possible that it will achieve its > karma requirement before libtalloc does. It would then be pushed to the > "fedora-updates" repository and then introduce a dependency issue in the > stable repository (because users trying to update libtevent would be > unable to update libtalloc without enabling the updates-testing > repository). > > The current recommended approach is to bundle the two updates into a > single one carrying multiple packages. The first problem with this is > that you must have commit privilege on all packages that you are > bundling into an update. If you do not, then you need to track down a > provenpackager to do it for you. > > Thank you for your post about this topic. In this year we have as a minimum two cases were we have got broken dependencies in the F-16 stable repository. 1.) alsa-utils and alsa-libs was pushed out of sync 2.) gcc and llvm which requires as specific version of gcc was pushed out of sync. The issue is, that the user will get an ugly error message which is not a good advertisement for the Fedora project. I have wrote mails to the maintainers and in the second case I have got the feeling, that the maintainer didn't understood my complain. Of course we can get run in error if the push fails due an technical issue, but this is an onther topic. Best Regards: Jochen Schmitt -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iJwEAQECAAYFAk9w0cYACgkQZLAIBz9lVu+lVAP9F50AXP0B5nDHWvFuxRd4f5wY UyZz2gkNi7CMX5jD9DkkzQ+swrrvEVOQCpUA4zOplP3bsEpR8MV8i4Y104gz8Ygw cCg/V8xhEMfalhJ3ORDST6yVLlBViHKsjOcV0tJ0fAFX10zShxXwlnxutp+RP8V2 MKT4JUhj1lD8knqWo50= =XZBT -END PGP SIGNATURE- -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Dependencies on Bodhi Updates
Stephen Gallagher wrote: > 2) We could continue on the "single update for multiple packages" > approach, but revamp the karma system so that each SRPM gets its own > karma, rather than the update as a whole. Then, the whole update would > not be pushed via autokarma until all of the dependent packages had > sufficient karma (or the owner of the update could push them after the > stable wait period, of course). This just does not scale for large update groups such as the KDE SC updates. I think it would also acerbate the already existing "How do I provide feedback for a library?" problem (because now it'd also affect libraries included in update groups, not just those filed separately). I don't understand why we need to make up more and more complicated rules for updates rather than just killing the whole karma and autokarma business and having the maintainer READ the update feedback and make an informed (and unhindered by bureaucracy) decision based on that. Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Dependencies on Bodhi Updates
As requested during the FESCo meeting, I am going to try to summarize some of the issues inherent in the way that Bodhi updates currently work. First, I'll try to explain the goals and constraints: 1) The stable 'fedora-updates' yum repository should NEVER exist in a state where any package has dependency issues. In other words, it should never be possible for an update to be pushed to stable that breaks cleanly updating any other package. 2) Updates must be possible and (ideally) timely. This is probably self-evident. 3) Packages pushed to the stable 'fedora-updates' yum repository should (ideally) not introduce regressions in packages that depend on them. 4) New features in "superpackages" such as Firefox, GNOME or FreeIPA that have many and varied dependencies may require new features in packages they depend on in order to enhance or fix the superpackage. In the trivial example, a package (let's say libtalloc) needs to make an update to fix a bug. This package requires nothing new from its dependencies and is a self-contained fix. For this example, it is simple to just build libtalloc in koji and then create a Bodhi update and pass it through "updates-testing", get karma and *poof* off to "fedora-updates". Now let's extend the example. Suppose that we have another package libtevent that has libtalloc as a dependency. Libtevent's maintainer wants to add a new feature to libtevent, but the patch from upstream depends on the bug in libtalloc having been fixed in order for the new feature to work properly. In this situation, the maintainer of libtevent would build libtevent with an explicit Requires: libtalloc >= in the specfile (possibly pulling libtalloc into the BuildRoot overrides if necessary) and then test it locally to see that it works. So now we have our first updates dependency issue. If we submit libtevent as its own update, it is possible that it will achieve its karma requirement before libtalloc does. It would then be pushed to the "fedora-updates" repository and then introduce a dependency issue in the stable repository (because users trying to update libtevent would be unable to update libtalloc without enabling the updates-testing repository). The current recommended approach is to bundle the two updates into a single one carrying multiple packages. The first problem with this is that you must have commit privilege on all packages that you are bundling into an update. If you do not, then you need to track down a provenpackager to do it for you. Now let's make the problem even more fun. Consider that the update to libtevent might be coming in because it is necessary for a new feature in libldb, which is in turn providing new functionality necessary for SSSD. So now we have four packages all sitting in the same update. The problem with this is that the tendency will be to only test the most user-visible package(s) in the set. In this particular case, that might be SSSD. So people would likely test SSSD and, if nothing went wrong, consider the entire update stable. But wait! SSSD isn't the only package that depends on libldb, libtevent and libtalloc. So too does the samba package. Suppose that the bugfix in libtalloc, after resolving the original issue, results in exposing another more serious bug in samba? Now we need to pull a samba update into this same update series. A contrived example, you say? That would never happen, bugfixes aren't likely to do that. Well, for one example: https://admin.fedoraproject.org/updates/FEDORA-2011-11845 In this particular example, we knew up-front that it was going to necessitate a rebuild of several dependent packages and we coordinated a single release to address them. So in this case, the proper approach was to bundle them together in a single update. This worked because we specifically knew that the libtevent change was going to break other packages. But what about when we don't know that? Let's take another example: https://admin.fedoraproject.org/updates/FEDORA-2011-17399 In this case, there was a security bug reported against Firefox. Such things are serious, and acted on quickly. However, the bug was actually fixed in the nss package, and Firefox, Xulrunner and friends were rebuilt against that nss package. The problem was this: the fix made to the nss package introduced regressions in every other package that depended on it. However, because the default install of Firefox contained no issues, it rapidly received the necessary karma points and the whole update was pushed to stable. It then broke nearly every application in Fedora that relied on cryptography. The problem here was sociological, not technological. The only package that received testing was Firefox. It's hard to say without evidence whether the problem would have been averted by having nss go through its own update, but I strongly suspect that what we would have seen was greater testing on actual nss features for that specific update. Of course, we now have the same p
