I thought this was exactly the JSON blob we will implement, If you look at the model of PDC https://product-definition-center.github.io/product-definition-center/_images/overview.svg
Those endpoints are covered by relations Release, Compose, RPM through some others. The PDC approach is more or less a normalized RDBMS model. This should be replaceable by a much simpler denormalized approach and create one entry for each compose with all the data in one place. It is also mentioned in the arc docs https://fedora-arc.readthedocs.io/en/latest/pdc/users.html#fedoraqa-fedfind On Tue, Sep 12, 2023 at 2:18 AM Adam Williamson <adamw...@fedoraproject.org> wrote: > On Mon, 2023-09-11 at 09:40 -0700, Kevin Fenzi wrote: > > These are the things that fedfind/qa users? Do we have examples of this > > data? > > Okay, here is (again) a list of the stuff fedfind uses. You can see > sample data for each compose in the web UI itself, PDC actually has a > rather good interactive API view. > > 1. WHAT: https://pdc.fedoraproject.org/rest_api/v1/composes/ > HOW: ?release=fedora-NN&compose_label=label > ?compose_id=cid > WHY: To find a compose's compose ID from its label, or a compose's > label from its compose ID. These functions are used on various paths. > The most important one in real life is probably when we are creating > and updating validation event pages for candidate composes, between the > time the compose exists and the time it is synced to alt.fp.o - during > this time wikitcms may need to look the compose up by label, but for > fedfind to actually find it, cid_from_label has to work, because it can > only 'natively' find a compose by label if it's been synced to > alt.fp.o; if the compose has not yet been synced, the only way fedfind > can find the compose is to use cid_from_label then locate it on > kojipkgs under its compose ID. > WORKAROUND: it would be difficult to work around this if we did not > have the ability. For wikitcms we could try 'embedding' the compose ID > in the event pages somehow. For some other less important uses there is > probably no workaround. > > 2. WHAT: https://pdc.fedoraproject.org/rest_api/v1/compose-images/ > HOW: compose-images/(composeid) > WHY: To provide image metadata for nightly composes that have been > garbage-collected, and releases in the mirror system after they have > been split across directories and their metadata stripped > WORKAROUND: if there is no store of metadata by compose ID then > fedfind just can't do this any more. It's not critical functionality to > anything important I'm aware of, though - it's just a nice feature that > can be used to e.g. run long-term analysis of image size changes across > multiple releases, stuff like that. If this goes away I will just drop > this feature from fedfind and you will no longer be able to find the > real metadata for these composes (it would provide synthesized metadata > for mirrored releases, but not garbage-collected nightlies). Note, > retrieving the original metadata for mirrored releases depends on the > cid_from_label feature (see 1). > > 3. WHAT: https://pdc.fedoraproject.org/rest_api/v1/releases/ > HOW: ?version=39&name=Fedora (for e.g.) > WHY: To find the compose that was "previous" to any given compose. > The first result for this PDC query for any given release is a dict > with a handy 'compose_set' key whose value is a list of all the > composes for that release, in reverse order by date. So we can easily > find the compose that came 'before' any given compose. This is, I > *think*, only used by the 'check-compose' script which sends out those > "compose check report" emails. > WORKAROUND: fedfind has a hideous alternative implementation of this > which involves just blindly trying possible decrements and seeing if > they exist. e.g. if you ask for the compose previous to Fedora-39- > 20230911.n.1, it will try 20230911.n.0, if that doesn't exist, it will > try 20230910.n.5 (we start counting backwards from 5 because, hey, > gotta start somewhere, and doing more than 5 composes on one day is > unlikely), then 20230910.n.4, and so on, until it hits something that > exists. So, if we can't get a nice data set like this any more...we'll > just have to use that. Bleh. We might also just drop check-compose and > kill the feature, I suppose. I don't pay as much attention to those > reports as I used to. > > 4. WHAT: https://pdc.fedoraproject.org/rest_api/v1/rpms/ > HOW: ?compose=cid&arch=src&name=^package1$&name=^package2$.... > WHY: To get the NEVR (name, epoch, version, release) for a given set > of packages from a compose. This can be used by relvalconsumer (the > thing that creates the wiki validation events) to decide whether any > "important" packages changed between the last tested compose and the > compose it's currently deciding whether to create a validation event > for: if it's been more than 3 days but less than 14 days since the last > event, it will create a new event *if* any important package's version > changed. > WORKAROUND: Looking back on the history of this, I don't think > anything's actually using it right now. There is an ugly alternative > version of this one, too, which scrapes dl.fedoraproject.org's HTML > output (told you it was ugly!). I initially replaced that version with > the PDC one for some compose types, but then ran into problems with > that, and ultimately decided to provide both side-by-side; I think > relvalconsumer is the only thing that actually uses this feature, and > it currently uses the ugly HTML scraping version, not the PDC version. > I probably meant to try and make it use the PDC version but never got > around to it. The ugly version is working okay in practice, so I > suppose the workaround here would just be to drop the pretty PDC > version and rely on HTML scraping forever (sob). > -- > Adam Williamson (he/him/his) > Fedora QA > Fedora Chat: @adamwill:fedora.im | Mastodon: @ad...@fosstodon.org > https://www.happyassassin.net > _______________________________________________ > infrastructure mailing list -- infrastructure@lists.fedoraproject.org > To unsubscribe send an email to > infrastructure-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org > Do not reply to spam, report it: > https://pagure.io/fedora-infrastructure/new_issue > -- Tomas Hrcka fas: humaton libera.CHAT: jednorozec
_______________________________________________ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
