Re: How can we make security updates faster?
On Tue, May 29, 2012 at 06:58:00PM +, "Jóhann B. Guðmundsson" wrote: > On 05/29/2012 06:13 PM, Rex Dieter wrote: > >>> It makes no sense to have a gui application ( or an application for that > >>> matter ) without having written the relevant how to debug/how to test > >>> pages for each component to accommodate it. > >Indeed. However, I'd argue*both* pieces, a karma app and good test-cases, > >are needed, and one not need block on the other. > > Well we then agree on disagreeing since updating the component and > running the relevant application is hardly what I call testing and > requiring karma points to just do that makes absolutely no sense to > me. It is called smoketesting and I think it fulfills the objective of avoiding broken updates pretty well, thanks. Note that we are talking about Fedora here, not RHEL. If you are not satisfied with the current level of testing, you are welcomed to do something about it. But forcing others to do something is not the right way to go. > In any case this is something we solve in the QA community and > arguably we should be the one that decide all this and FPC just > implements what we have decided and tell them to. > > This really does not involve Fesco and we already have a good > working relationship with releng. Again, this is Fedora we are talking about, not RHEL. If you (the QA community) decide on some new policy that most maintainers disagree with, they will just ignore it and there is _absolutely nothing_ you can do to enforce it. Period. D. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
On Tue, May 29, 2012 at 05:46:38AM +, "Jóhann B. Guðmundsson" wrote: > On 05/29/2012 05:21 AM, Adam Williamson wrote: > >We actually have this on the QA wishlist and it was one of the projects > >we proposed for GSoC for QA, but it didn't quite make it. We may still > >wind up doing it through some other channel, though. See also > >https://fedoraproject.org/wiki/Summer_coding_ideas_for_2012#Fedora_Gooey_Karma > > andhttp://blog.tirfa.com/gooey-karma. > > It makes no sense to have a gui application ( or an application for > that matter ) without having written the relevant how to debug/how > to test pages for each component to accommodate it. > > Such an application will just fail without it and might cause more > harm then good and yeah I have already mention the root cause for > those pages not already existing on this thread. I could not disagree more. It is blind reliance on written test cases that might cause more harm than good. IOW, if a tester does not know how a package works, it would be better if he did not "test" it. D. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
On Tue, 29 May 2012 21:56:37 + "Jóhann B. Guðmundsson" wrote: > On 05/29/2012 07:49 PM, Kevin Fenzi wrote: > > Bodhi does. > > https://admin.fedoraproject.org/updates/ > > > > F17 security updates: 122 > > F16 security updates: 310 > > F15 security updates: 444 > > > > Luke probably has previous releases historical data somewhere. > > That means ca 2 hours per day spent in testing for all GA releases... Don't mean to dispute your numbers, but could you explain how you arrived at that? kevin signature.asc Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
- Original Message - > Jóhann B. Guðmundsson wrote: > > > On 05/29/2012 05:21 AM, Adam Williamson wrote: > >> We actually have this on the QA wishlist and it was one of the > >> projects > >> we proposed for GSoC for QA, but it didn't quite make it. We may > >> still > >> wind up doing it through some other channel, though. See also > >> > https://fedoraproject.org/wiki/Summer_coding_ideas_for_2012#Fedora_Gooey_Karma > >> andhttp://blog.tirfa.com/gooey-karma. > > > > It makes no sense to have a gui application ( or an application for > > that > > matter ) without having written the relevant how to debug/how to > > test > > pages for each component to accommodate it. > > Indeed. However, I'd argue *both* pieces, a karma app and good > test-cases, > are needed, and one not need block on the other. Security updates are usually a different sort of beasts - even as developer you sometimes do not have a way how to reproduce it, or steps are not very clear (as for example some parts are not disclosed etc.). Also sometimes we need to go out with an update to Bodhi while it's still under EMBARGO to be able to release it once embargo is over (so it should be visible only for a specific group of people, you can't disclose it on public, not a test case...). So something like not only Security SIG but our own Fedora Sec. response team makes sense. But it's a huge amount of work - so it's only for brave people... You have to work with developers from the beginning to the release, it's also coordination job etc. Jaroslav > -- rex > > -- > devel mailing list > devel@lists.fedoraproject.org > https://admin.fedoraproject.org/mailman/listinfo/devel -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
On 05/29/2012 07:49 PM, Kevin Fenzi wrote: Bodhi does. https://admin.fedoraproject.org/updates/ F17 security updates: 122 F16 security updates: 310 F15 security updates: 444 Luke probably has previous releases historical data somewhere. That means ca 2 hours per day spent in testing for all GA releases... JBG -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
re: How can we make security updates faster?
Hi, I have a suggestion, not totally related. It would be nice to have a tool which does the same thing than portaudit for FreeBSD. This tool is simple: you launch it, and it lists which packages are vulnerable. That's way you don't need to wait for a package to be in -testing or in -stable to know whether there is a security issue. It could improve tests also. Because if the tool lists a package which is vulnerable, if it is in -testing and not yet in -stable, then more users will update it from testing. I did not reply and created a new subject because I was not subscribed to the list. Documentation: freebsd.org/doc/handbook/security-portaudit.html -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
On Tue, 29 May 2012 19:42:21 + "Jóhann B. Guðmundsson" wrote: > On 05/29/2012 07:27 PM, Kevin Fenzi wrote: > > It wouldn't have to be I wouldn't think... they could also test for > > general functionality or serious regressions. > > Does infrastructure keep somewhere statistic how many security > updates we push per release cycles so we can roughly calculate how > much manpower it takes from the security team to effectively pull > this off? Bodhi does. https://admin.fedoraproject.org/updates/ F17 security updates: 122 F16 security updates: 310 F15 security updates: 444 Luke probably has previous releases historical data somewhere. kevin signature.asc Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
On 05/29/2012 07:27 PM, Kevin Fenzi wrote: It wouldn't have to be I wouldn't think... they could also test for general functionality or serious regressions. Does infrastructure keep somewhere statistic how many security updates we push per release cycles so we can roughly calculate how much manpower it takes from the security team to effectively pull this off? JBG -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
On Tue, 29 May 2012 19:15:21 + "Jóhann B. Guðmundsson" wrote: > On 05/29/2012 06:39 PM, Kevin Fenzi wrote: > > Perhaps if there's enough interest we could (re)vive a Security SIG > > of some kind? One of their goals could be to cross test updates and > > provide karma? > > Would not their participation be more geared to test if the exploit > has actually been closed rather then performing general testing of > the application and if the update breaks anything else? It wouldn't have to be I wouldn't think... they could also test for general functionality or serious regressions. kevin signature.asc Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
On 05/29/2012 06:39 PM, Kevin Fenzi wrote: Perhaps if there's enough interest we could (re)vive a Security SIG of some kind? One of their goals could be to cross test updates and provide karma? Would not their participation be more geared to test if the exploit has actually been closed rather then performing general testing of the application and if the update breaks anything else? JBG -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
On 05/29/2012 06:13 PM, Rex Dieter wrote: > It makes no sense to have a gui application ( or an application for that > matter ) without having written the relevant how to debug/how to test > pages for each component to accommodate it. Indeed. However, I'd argue*both* pieces, a karma app and good test-cases, are needed, and one not need block on the other. Well we then agree on disagreeing since updating the component and running the relevant application is hardly what I call testing and requiring karma points to just do that makes absolutely no sense to me. This whole scenario is a bit more complicated than that and me and James did discuss few ideas to solve this when he was with the project but things did not progress any further than that for various reasons including the FPC/FESCO decisions to make things optional instead of mandatory. For security updates arguably we should be having faith in maintainers actually eating and testing their own food and use a time based limit instead of karma based as in after x many days in updates testing and no reporter has reported any problem the update gets pushed regardless of it's karma. In any case this is something we solve in the QA community and arguably we should be the one that decide all this and FPC just implements what we have decided and tell them to. This really does not involve Fesco and we already have a good working relationship with releng. JBG -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
On Mon, 28 May 2012 12:57:18 -0400 (EDT) Paul Wouters wrote: > > Hi, > > I've recently had release updates to two packages with CVE issues in > then. A few weeks ago, pidgin-otr needed a lot of me prodding people > to try it and give karma to get the security update out. Right now, my > socat CVE security releases sits in all four branches with no karma > after four days. > > Is there something we can do to make these security updates move > faster? ...snip... > Any other thoughts? Perhaps if there's enough interest we could (re)vive a Security SIG of some kind? One of their goals could be to cross test updates and provide karma? kevin signature.asc Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
On Tue, 29 May 2012 13:13:43 -0500 Rex Dieter wrote: > Jóhann B. Guðmundsson wrote: > > > On 05/29/2012 05:21 AM, Adam Williamson wrote: > >> We actually have this on the QA wishlist and it was one of the > >> projects we proposed for GSoC for QA, but it didn't quite make it. > >> We may still wind up doing it through some other channel, though. > >> See also > >> > https://fedoraproject.org/wiki/Summer_coding_ideas_for_2012#Fedora_Gooey_Karma > >> andhttp://blog.tirfa.com/gooey-karma. > > > > It makes no sense to have a gui application ( or an application for > > that matter ) without having written the relevant how to debug/how > > to test pages for each component to accommodate it. > > Indeed. However, I'd argue *both* pieces, a karma app and good > test-cases, are needed, and one not need block on the other. I agree. I'll also note for non gui use 'fedora-easy-karma' works great. kevin signature.asc Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
Jóhann B. Guðmundsson wrote: > On 05/29/2012 05:21 AM, Adam Williamson wrote: >> We actually have this on the QA wishlist and it was one of the projects >> we proposed for GSoC for QA, but it didn't quite make it. We may still >> wind up doing it through some other channel, though. See also >> https://fedoraproject.org/wiki/Summer_coding_ideas_for_2012#Fedora_Gooey_Karma >> andhttp://blog.tirfa.com/gooey-karma. > > It makes no sense to have a gui application ( or an application for that > matter ) without having written the relevant how to debug/how to test > pages for each component to accommodate it. Indeed. However, I'd argue *both* pieces, a karma app and good test-cases, are needed, and one not need block on the other. -- rex -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
On 05/29/2012 05:21 AM, Adam Williamson wrote: We actually have this on the QA wishlist and it was one of the projects we proposed for GSoC for QA, but it didn't quite make it. We may still wind up doing it through some other channel, though. See also https://fedoraproject.org/wiki/Summer_coding_ideas_for_2012#Fedora_Gooey_Karma andhttp://blog.tirfa.com/gooey-karma. It makes no sense to have a gui application ( or an application for that matter ) without having written the relevant how to debug/how to test pages for each component to accommodate it. Such an application will just fail without it and might cause more harm then good and yeah I have already mention the root cause for those pages not already existing on this thread. JBG -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
On Mon, 2012-05-28 at 23:49 +0200, Michael Scherer wrote: > Le lundi 28 mai 2012 à 12:57 -0400, Paul Wouters a écrit : > > Hi, > > > > I've recently had release updates to two packages with CVE issues in > > then. A few weeks ago, pidgin-otr needed a lot of me prodding people > > to try it and give karma to get the security update out. Right now, my > > socat CVE security releases sits in all four branches with no karma after > > four days. > > > > Is there something we can do to make these security updates move faster? > > > > Perhaps a new mailinglist that just announces the security releases, to > > remind people to test them and give karma. > > > > Perhaps a gui app for people running post latest full release fedora > > installs that checks if some software you are using is in need of karma? > > I would take this road. We actually have this on the QA wishlist and it was one of the projects we proposed for GSoC for QA, but it didn't quite make it. We may still wind up doing it through some other channel, though. See also https://fedoraproject.org/wiki/Summer_coding_ideas_for_2012#Fedora_Gooey_Karma and http://blog.tirfa.com/gooey-karma. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
On 05/28/2012 08:35 PM, Paul Wouters wrote: The point of a seperate list would be that peopel interested in giving security updates some extra attention wouldn't be swamped with other emails, causing them just to filter and file those emails unseen. If the pidgin-otr and socat security update information ended up going to any QA related lists, it did not seem to help. They do so via the update-testing report and even up on top to be the first thing reporters read so getting the information to reporters is not the problem but getting them to actually test components is and there are several issues we need to solve in that regard. I did an honest effort to improve that situation in the past ( the whole scenario is bit more complicated ) when we had around 6000 components in the distribution but members of FPC/FESCO choice to make something that was necessary for us ( QA ) to be mandatory to solve that and other problems, optimal. So instead of having the roughly 6000 components that have been added since then plus what we could have caught up with of those existing 6000 components with what is needed now, we still have roughly the effort I did until I decided to drop it altogether since it was quite foreseeable for people that have the ability to think further then their nose that it would never work with it being optional. Long story put short using "Karma" ain't working and wont work unless some serious effort is done to get that concept in a workable shape to at least give that concept an hope to ever potential to work in the first place. At this point in time we might just as well try some alternative solution to the task at hand instead of trying to patch the broken existing one. In any case that's something for us in QA to figure out and hopefully FPC/FESCO willl work with us instead of against us this time but given how some of them have acted and voted on various task they have been given for the last release cycle I doubt that's the case but elections are coming up so there might be hope there yet you know fresh blood,fresh minds,fresh ideas, fresh approaches etc... JBG -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
Le lundi 28 mai 2012 à 12:57 -0400, Paul Wouters a écrit : > Hi, > > I've recently had release updates to two packages with CVE issues in > then. A few weeks ago, pidgin-otr needed a lot of me prodding people > to try it and give karma to get the security update out. Right now, my > socat CVE security releases sits in all four branches with no karma after > four days. > > Is there something we can do to make these security updates move faster? > > Perhaps a new mailinglist that just announces the security releases, to > remind people to test them and give karma. > > Perhaps a gui app for people running post latest full release fedora > installs that checks if some software you are using is in need of karma? I would take this road. in fact, one issue I have with update is that to see if there is something interesting to test, I go to : https://admin.fedoraproject.org/updates/F17/testing First page is usually useless for this task, packages are not signed and not on mirror either, and I prefer to take the easiest road of using yum. 2nd page is having the same problem usually, so i need to start looking at the 3rd page to see testable packages but sometimes not. Then I need to look at every package, see if there is one that I can test either because it sound interesting, or because I use it. If the package is new, I click on it see the update, and then click again on the package name, to get to a page where i click to see a list of update, and a list of link, and one to the description of the package either pkgdb, or community. And if I want to see the website of the package, i need to google. That's too much click just to see something to test. And I still didn't installed it yet, and due to various mirrors lag, it sometimes doesn't work and so I forget. The same goes for any notification list or for bugzilla. When I receive notification, the package is not yet installable, so I forget. So yes, there need to have a way to connect people that care of a software up to the point of testing it, and karma. Being able to say "warn me if there is a new package to test of $FOO", and having a notification ( popup, email, whatever ) would surely help. And a reminder to give karma ( again, a popup after 1 day, saying "have you tested this, does it work [yes] [ask me later] [do not ask me again] ", something like fedora-easy-karma would be enough ) Taking only in account package in updates-testing indexes, this would remove the mirror lag issue. -- Michael Scherer -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
On Mon, 28 May 2012, "Jóhann B. Guðmundsson" wrote: Perhaps a new mailinglist that just announces the security releases, to remind people to test them and give karma. We already have a list that all test related information is supposed to go to including security related ones, in fact all QA related topic in general is supposed to go to that list since the whole QA community resides on that list but you and others can continue to post QA related topics to this list and expect magic to happen just don't be disappointed when nothing happens et all. The point of a seperate list would be that peopel interested in giving security updates some extra attention wouldn't be swamped with other emails, causing them just to filter and file those emails unseen. If the pidgin-otr and socat security update information ended up going to any QA related lists, it did not seem to help. Paul -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How can we make security updates faster?
On 05/28/2012 04:57 PM, Paul Wouters wrote: Perhaps a new mailinglist that just announces the security releases, to remind people to test them and give karma. We already have a list that all test related information is supposed to go to including security related ones, in fact all QA related topic in general is supposed to go to that list since the whole QA community resides on that list but you and others can continue to post QA related topics to this list and expect magic to happen just don't be disappointed when nothing happens et all. JBG -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
How can we make security updates faster?
Hi, I've recently had release updates to two packages with CVE issues in then. A few weeks ago, pidgin-otr needed a lot of me prodding people to try it and give karma to get the security update out. Right now, my socat CVE security releases sits in all four branches with no karma after four days. Is there something we can do to make these security updates move faster? Perhaps a new mailinglist that just announces the security releases, to remind people to test them and give karma. Perhaps a gui app for people running post latest full release fedora installs that checks if some software you are using is in need of karma? Perhaps push security releases within 3 days if no -1 karma has been received? Push updates to stable when a certain download/install ratio has been seen with no -1 karma? Any other thoughts? Paul -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
