Re: URGENT: users prompted to upgrade to F32
On Sat, 2020-02-22 at 12:42 -0800, Kevin Fenzi wrote: > On Tue, Feb 18, 2020 at 09:33:42PM -0700, Ken Dreyer wrote: > > On Mon, Feb 17, 2020 at 10:52 AM Adam Williamson > > wrote: > > > > > Since this is an API endpoint of a real system which needs to be > > > updated correctly when the release events actually happen, it > > > should > > > have the benefits pkgdb used to be (the information should be > > > reliable > > > and timely) > > > > Do we have stats on how many hits per second that current endpoint > > receives? > > It looks like it gets less than 100k/day. So, one or less per second. Are they evenly-spaced, though? I don't recall exactly how fancy the GNOME Software code is, but it seems at least possible that these check-ins aren't evenly spaced out but come in clumps... BTW, I've ported fedfind from collections to Bodhi in 4.4.0, 4.4.1 is in updates-testing now. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: URGENT: users prompted to upgrade to F32
On Tue, Feb 18, 2020 at 09:33:42PM -0700, Ken Dreyer wrote: > On Mon, Feb 17, 2020 at 10:52 AM Adam Williamson > wrote: > > > Since this is an API endpoint of a real system which needs to be > > updated correctly when the release events actually happen, it should > > have the benefits pkgdb used to be (the information should be reliable > > and timely) > > Do we have stats on how many hits per second that current endpoint receives? It looks like it gets less than 100k/day. So, one or less per second. > > It would be a bummer to inadvertently bring down Bodhi because of this > feature. Agreed. We should definitely make sure if we switch to this it can handle the load, but it doesn't seem like it will be much of a problem. > > What do you think about having Bodhi write out a flat file to disk or > something for Apache to serve in a similar way, so it doesn't have to > go through mod_wsgi or touch the database for that URL? We could, but I don't think it gets enough traffic to warrent that... Something to discuss with the Bodhi development team. kevin signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: URGENT: users prompted to upgrade to F32
On Mon, Feb 17, 2020 at 10:52 AM Adam Williamson wrote: > Since this is an API endpoint of a real system which needs to be > updated correctly when the release events actually happen, it should > have the benefits pkgdb used to be (the information should be reliable > and timely) Do we have stats on how many hits per second that current endpoint receives? It would be a bummer to inadvertently bring down Bodhi because of this feature. What do you think about having Bodhi write out a flat file to disk or something for Apache to serve in a similar way, so it doesn't have to go through mod_wsgi or touch the database for that URL? - Ken ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: URGENT: users prompted to upgrade to F32
On Mon, 2020-02-17 at 09:51 -0800, Adam Williamson wrote: > > However! It seems that there *may* be an alternative candidate for the > role. Bodhi has a 'releases' API endpoint, here: > > https://bodhi.fedoraproject.org/releases/ I updated a gnome-software issue: https://gitlab.gnome.org/GNOME/gnome-software/issues/214 and filed a fedfind issue: https://pagure.io/fedora-qa/fedfind/issue/17 if anyone's aware of any other systems using the collections data, please flag them up so we can consider those too. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: URGENT: users prompted to upgrade to F32
On Sun, 2020-02-16 at 22:17 +0100, drago01 wrote: > On Saturday, February 15, 2020, Kevin Fenzi wrote: > > > On Fri, Feb 14, 2020 at 05:14:57PM -0800, Adam Williamson wrote: > > > On Fri, 2020-02-14 at 18:37 -0600, Michael Catanzaro wrote: > > > > Hi, > > > > > > > > Users are being prompted to upgrade to F32. Anyone who knows how to > > fix > > > > this, please help with > > > > https://pagure.io/fedora-infrastructure/issue/8652 ASAP. > > > > > > We fixed it several hours ago, but people are likely hitting copies of > > > the bad data cached somewhere along the line :/ > > > > > > https://pagure.io/fedora-infrastructure/issue/8652#comment-626587 > > > > Yeah, it was live for about 45min this morning, but we fixed it as soon > > as it was noted. ;( > > > > Not sure that there is too much more we can do at this point... > > > > > What can we do to ensure it does not happen again? Well, ideally, finally come up with an actually sane system / set of processes for handling release events and storing/providing information about them. Unfortunately, as Kevin puts it, we could do a great job of that if someone would give us four people who could work on it for six months. As long as that doesn't happen, releng will keep putting duct tape on duct tape, and when resource constraints force people to do that, this kind of accident will happen. In practice, though, there might be something concrete we can do in the short term. The problem here is that GNOME Software (and fedfind, as it happens) are using this thing as a Source of Truth about Fedora releases: https://admin.fedoraproject.org/pkgdb/api/collections/ when pkgdb existed, that was one of its API endpoints. It could be relied upon to be a reliable source of information about Fedora releases that existed and their status (stable, development, EOL) because it *had* to be updated accurately at each 'lifecycle event' (Branching, EOL, stable release) or else various other things wouldn't work right. But pkgdb was retired in 2017. Since then this URL has not been a reliable API endpoint, but a weird orphan: it's literally a JSON format text file which a web server is configured to serve out at that URL. At each 'lifecycle event', someone has to update the text file by hand, and do it right. What happened in this case is someone made a mistake while doing that - I'm not going to say who, because of course it could happen to anyone as long as this fundamentally silly process is in place. We keep this silly process in place because there hasn't really been a viable alternative to it. PDC was *supposed* to be the replacement for pkgdb so far as 'Official Source of Truth for release information' went, but for a variety of reasons - significantly https://github.com/product-definition-center/product-definition-center/issues/294 - it never really has been. Now RH has lost interest in PDC, Fedora is in an awkward spot where we're sort of still using it, but don't have the resources to really complete all the plans for its adoption. However! It seems that there *may* be an alternative candidate for the role. Bodhi has a 'releases' API endpoint, here: https://bodhi.fedoraproject.org/releases/ (it renders as HTML in a browser, but you get JSON when accessing it in API-y ways). I think it actually provides the information that at least GNOME Software and fedfind need - 'these are the releases that exist, and their lifecycle status is X'. One key question is exactly when Fedora 32 becomes 'current' for Bodhi's purposes - it needs to be at the time we actually decide Fedora 32 is stable, not at the Bodhi activation point or the point when we cut the final fedora-release package or anything like that. Another possible issue is distinguishing Branched from Rawhide. But if the 'stable release' event happens at the right time, and we can distinguish Branched and Rawhide somehow, I think we may have a good candidate. Since this is an API endpoint of a real system which needs to be updated correctly when the release events actually happen, it should have the benefits pkgdb used to be (the information should be reliable and timely), and it shouldn't be as vulnerable to PEBKAC events as...a hand-edited JSON text file. So if we update GNOME Software and fedfind, and anything else using collections as a source of truth, to use this Bodhi API instead, potentially that could help. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: URGENT: users prompted to upgrade to F32
On Saturday, February 15, 2020, Kevin Fenzi wrote: > On Fri, Feb 14, 2020 at 05:14:57PM -0800, Adam Williamson wrote: > > On Fri, 2020-02-14 at 18:37 -0600, Michael Catanzaro wrote: > > > Hi, > > > > > > Users are being prompted to upgrade to F32. Anyone who knows how to > fix > > > this, please help with > > > https://pagure.io/fedora-infrastructure/issue/8652 ASAP. > > > > We fixed it several hours ago, but people are likely hitting copies of > > the bad data cached somewhere along the line :/ > > > > https://pagure.io/fedora-infrastructure/issue/8652#comment-626587 > > Yeah, it was live for about 45min this morning, but we fixed it as soon > as it was noted. ;( > > Not sure that there is too much more we can do at this point... > > What can we do to ensure it does not happen again? ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: URGENT: users prompted to upgrade to F32
On Friday, February 14, 2020 5:37:53 PM MST Michael Catanzaro wrote: > Hi, > > Users are being prompted to upgrade to F32. Anyone who knows how to fix > this, please help with > https://pagure.io/fedora-infrastructure/issue/8652 ASAP. A quick note: This only affected GNOME users, or those using GNOME Software. KDE users did not get that notification. -- John M. Harris, Jr. Splentity ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: URGENT: users prompted to upgrade to F32
On Fri, Feb 14, 2020 at 05:14:57PM -0800, Adam Williamson wrote: > On Fri, 2020-02-14 at 18:37 -0600, Michael Catanzaro wrote: > > Hi, > > > > Users are being prompted to upgrade to F32. Anyone who knows how to fix > > this, please help with > > https://pagure.io/fedora-infrastructure/issue/8652 ASAP. > > We fixed it several hours ago, but people are likely hitting copies of > the bad data cached somewhere along the line :/ > > https://pagure.io/fedora-infrastructure/issue/8652#comment-626587 Yeah, it was live for about 45min this morning, but we fixed it as soon as it was noted. ;( Not sure that there is too much more we can do at this point... kevin signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: URGENT: users prompted to upgrade to F32
On Fri, 2020-02-14 at 18:37 -0600, Michael Catanzaro wrote: > Hi, > > Users are being prompted to upgrade to F32. Anyone who knows how to fix > this, please help with > https://pagure.io/fedora-infrastructure/issue/8652 ASAP. We fixed it several hours ago, but people are likely hitting copies of the bad data cached somewhere along the line :/ https://pagure.io/fedora-infrastructure/issue/8652#comment-626587 -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
URGENT: users prompted to upgrade to F32
Hi, Users are being prompted to upgrade to F32. Anyone who knows how to fix this, please help with https://pagure.io/fedora-infrastructure/issue/8652 ASAP. Michael ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
