Re: Fedora and PDC, road forward
On Tue, 12 Jun 2018 at 16:48, Pierre-Yves Chibon wrote: > > Good Morning Everyone, > > So yesterday we held a meeting on #fedora-apps about the future of PDC in > Fedora. > We kept notes in etherpad at: http://etherpad.osuosl.org/pdc_fedora > Here is the gist of it: > > What do we currently store in PDC: > * modules data - the list of what modules have been built, what rpms are in > them, > and which ones are active or not. > * Stream branches, branch ownership, retirement dates (EOL/SLE) > * release/life-circle tracking >* product and product versions (fedpkg gets active Fedora and EPEL releases > from product versions endpont) > * critpath boolean on packages (bodhi uses this) > * metadata from composes (RPMS/images) > * "dependency" data about which rpms depend on which other rpms and which > containers include other rpms. This is not used by anything and can be > de-prioritized, dropped. (It was originally going to be used in > Freshmaker.) > Endpoints: > * release-component-relationships > * release-components > * List of all packages: fedora-packages uses this list to know what to index. > It sets up the "for loop" from which fedora-packages pulls data from all > our > other systems. >Endpoints: >* global-components > > > After some discussion the decision we reached was: > * The "dependency" data in its current form can be dropped. It's something > we'll > need in the future but the data structure will most likely need to be > adjusted so let's just drop the existing data and worry about this later. > * The critpath boolean can just be imported into bodhi itself, especially > considering that bodhi is the only tool using this flag. During our last infrastructure meeting [0], tflink mentioned that the critpath boolean was also used by taskotron. Currently taskotron still uses pkgdb to get this info, so taskotron will need to be updated to use the new source of truth for the critpathwe might reconsider moving this to bodhi and leave in the PDC replacement. [0] - https://meetbot.fedoraproject.org/teams/infrastructure/infrastructure.2018-06-28-14.00.log.html > * The modules data is entirely going to move into MBS > > Everything else (ie: composes metadata, release/life-circle tracking, > package/branches information) will need to find a new home. > This new home will be a new django app using the Django Rest Framework (DRF) > in > which we will import the PDC apps as needed (potentially adjusting them where > desired). > > - kellin has agreed to be the project leader for this work but needs a team to > do the actual work. > - pingou has agreed to look for said team > > > If you have any worries, questions or suggestions or if you want to join the > effort, please let us know :) > > > Finally some links: > Minutes: > https://meetbot.fedoraproject.org/fedora-apps/2018-06-11/pdc_and_fedora.2018-06-11-14.01.html > Minutes (text): > https://meetbot.fedoraproject.org/fedora-apps/2018-06-11/pdc_and_fedora.2018-06-11-14.01.txt > Log: > https://meetbot.fedoraproject.org/fedora-apps/2018-06-11/pdc_and_fedora.2018-06-11-14.01.log.html > > > Have a nice day, > Pierre > ___ > infrastructure mailing list -- infrastructure@lists.fedoraproject.org > To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/N5UHPQE4NZO6O4VZK7WPFWPWVSSN3367/ ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/Y2U2Z2PEGUWP72XCPPTI7W66NL7NEYLR/
Re: Fedora and PDC, road forward
On Tue, Jun 19, 2018 at 09:39:25AM +0200, Aurelien Bompard wrote: > >> https://github.com/noirbizarre/flask-restplus > > Actually, I haven't tried that one. It seems pretty good (from the > docs), has anybody tried it? It does expose API doc using swagger as does the Connexion extension Clément just pointed to. Pierre ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/2QXELFP6ALFWSDZ3UGMR5TS5M6364C3G/
Re: Fedora and PDC, road forward
On Tue, Jun 19, 2018 at 09:39:25AM +0200, Aurelien Bompard wrote: > >> https://github.com/noirbizarre/flask-restplus > > Actually, I haven't tried that one. It seems pretty good (from the > docs), has anybody tried it? Seen it but not tried it :( Pierre ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/EDIK5YVHL7IC5GPYTURY3KTN2WAIWWWI/
Re: Fedora and PDC, road forward
On Tue, Jun 19, 2018 at 08:35:54AM +0200, Clement Verna wrote: > On Tue, 19 Jun 2018 at 01:13, Aurelien Bompard > wrote: > > > > > Within limits. It should be a version thats supported and gets at least > > > security updates. Hopefully the one(s) in Fedora follow this. > > > > Yeah it's 1.11 now which is LTS, since it'll be the last version to > > support Python 2 > > > > > There are a few flask rest frameworks, but I have not much idea how well > > > they are supported or work. > > > https://github.com/flask-restful/flask-restful > > > https://github.com/noirbizarre/flask-restplus > > > > Yeah I tried those in my search for something like DRF in the Flask > > world. They are decent, but far from DRF feature-wise. > > I still think we should keep Django, but I came across this tutorial > [0] yesterday, and it looks like this is a neat way of dealing with > REST API using Flask and connexion [1] > > [0] - https://realpython.com/flask-connexion-rest-api/ > [1] - https://github.com/zalando/connexion The tutorial is quite tempting, it looks quite nice, but the entire auth aspect isn't covered (though we do have flask-oidc). Pierre ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/ZP2OZLO3FEK6PJOTHKJU5JGEXFZBEMUJ/
Re: Fedora and PDC, road forward
>> https://github.com/noirbizarre/flask-restplus Actually, I haven't tried that one. It seems pretty good (from the docs), has anybody tried it? ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/CYAFYO2OUY4TZKXOPOX2WTUFRIOWGN5O/
Re: Fedora and PDC, road forward
On Tue, 19 Jun 2018 at 01:13, Aurelien Bompard wrote: > > > Within limits. It should be a version thats supported and gets at least > > security updates. Hopefully the one(s) in Fedora follow this. > > Yeah it's 1.11 now which is LTS, since it'll be the last version to > support Python 2 > > > There are a few flask rest frameworks, but I have not much idea how well > > they are supported or work. > > https://github.com/flask-restful/flask-restful > > https://github.com/noirbizarre/flask-restplus > > Yeah I tried those in my search for something like DRF in the Flask > world. They are decent, but far from DRF feature-wise. I still think we should keep Django, but I came across this tutorial [0] yesterday, and it looks like this is a neat way of dealing with REST API using Flask and connexion [1] [0] - https://realpython.com/flask-connexion-rest-api/ [1] - https://github.com/zalando/connexion > > A. > ___ > infrastructure mailing list -- infrastructure@lists.fedoraproject.org > To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/N7PGWQ3O252OGT7JLKFWYINAA56AM43P/ ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/UH2ROTQIGY4ODNSUARZJF4NRACSG2IDA/
Re: Fedora and PDC, road forward
> Within limits. It should be a version thats supported and gets at least > security updates. Hopefully the one(s) in Fedora follow this. Yeah it's 1.11 now which is LTS, since it'll be the last version to support Python 2 > There are a few flask rest frameworks, but I have not much idea how well > they are supported or work. > https://github.com/flask-restful/flask-restful > https://github.com/noirbizarre/flask-restplus Yeah I tried those in my search for something like DRF in the Flask world. They are decent, but far from DRF feature-wise. A. ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/N7PGWQ3O252OGT7JLKFWYINAA56AM43P/
Re: Fedora and PDC, road forward
On 06/18/2018 01:25 PM, Aurelien Bompard wrote: >> I'm a little worried about Django. True, we have to maintain a version >> for mailman3, but it's rhel7/python3. Is this new app going to use that? > > Actually, HyperKitty and Postorius are using Django on Python 2.7. The > Django version is 1.8 and it's pretty old now. > I would recommend against starting a new app on Python 2 today and it > does not look like we have a Python 3 package of Django in EPEL yet. Yeah, this is the part that worries me... Django 1.8 went out of support on April 1st (no joke!). >> Alternately if we use Fedora, we need to adjust to new Django versions >> pretty often (one problem we already hit with PDC). > > Would it make sense to run it in OpenShift? I'd think so. Then we > could build it with whichever version works, right? Within limits. It should be a version thats supported and gets at least security updates. Hopefully the one(s) in Fedora follow this. >> Since this is just a simple api, could we do something more simple? > > The thing is that the Django REST Framework library is really > wonderful and there is no Flask equivalent that I know of. It would > save us handling of a lot of corner cases, and it has built-in tools > for versionning the API and thus preserving API compatibility. > Authentication is also very flexible, etc. It's nice. > > That said, nothing impossible to do in Flask, just longer and possibly > more error-prone. Yeah, understood. I'd just like to make sure we have security support and aren't leaving ourselves exposed. :( There are a few flask rest frameworks, but I have not much idea how well they are supported or work. https://github.com/flask-restful/flask-restful https://github.com/noirbizarre/flask-restplus kevin signature.asc Description: OpenPGP digital signature ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/AQRER4FRC6A277UPGSHVL6VWWTIQYF4C/
Re: Fedora and PDC, road forward
> I'm a little worried about Django. True, we have to maintain a version > for mailman3, but it's rhel7/python3. Is this new app going to use that? Actually, HyperKitty and Postorius are using Django on Python 2.7. The Django version is 1.8 and it's pretty old now. I would recommend against starting a new app on Python 2 today and it does not look like we have a Python 3 package of Django in EPEL yet. > Alternately if we use Fedora, we need to adjust to new Django versions > pretty often (one problem we already hit with PDC). Would it make sense to run it in OpenShift? I'd think so. Then we could build it with whichever version works, right? > Since this is just a simple api, could we do something more simple? The thing is that the Django REST Framework library is really wonderful and there is no Flask equivalent that I know of. It would save us handling of a lot of corner cases, and it has built-in tools for versionning the API and thus preserving API compatibility. Authentication is also very flexible, etc. It's nice. That said, nothing impossible to do in Flask, just longer and possibly more error-prone. Aurélien ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/UO2X3XYQUXXCCB45M5FNR2FSX6HVWPED/
Re: Fedora and PDC, road forward
On Wed, 2018-06-13 at 17:06 -0700, Kevin Fenzi wrote: > On 06/13/2018 03:19 PM, Adam Williamson wrote: > > On Wed, 2018-06-13 at 12:03 -0700, Kevin Fenzi wrote: > > > On 06/12/2018 07:41 AM, Pierre-Yves Chibon wrote: > > > > Good Morning Everyone, > > > > > > > > So yesterday we held a meeting on #fedora-apps about the future of PDC > > > > in Fedora. > > > > We kept notes in etherpad at: http://etherpad.osuosl.org/pdc_fedora > > > > Here is the gist of it: > > > > > > > > What do we currently store in PDC: > > > > * modules data - the list of what modules have been built, what rpms > > > > are in them, > > > > and which ones are active or not. > > > > * Stream branches, branch ownership, retirement dates (EOL/SLE) > > > > * release/life-circle tracking > > > >* product and product versions (fedpkg gets active Fedora and EPEL > > > > releases > > > > from product versions endpont) > > > > > > As an aside did we ever get that 'in development' addition we wanted so > > > we could point everything still using pkgdb to pdc for release status? > > > > All the code got done but no-one has yet/ever put a process in place to > > actually populate Fedora's PDC with the appropriate data, so it's not > > there. Things (fedfind, gnome-software...) are still using the > > 'collections' JSON for now. > > Is that the last thing using pkgdb that we know of? > > It would be pretty sad to keep running pkgdb, and pdc and the new thing > all at the same time. :( It's the last thing *I* know of. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/NZ3DGHONI7TFTX6AS6C2SPDHYRMW5WMB/
Re: Fedora and PDC, road forward
On Wed, 2018-06-13 at 17:04 -0700, Kevin Fenzi wrote: > On 06/13/2018 02:50 PM, Randy Barlow wrote: > > On 06/13/2018 03:03 PM, Kevin Fenzi wrote: > > > ok. Note that this data changes over time, and releng needs a script to > > > update it (or better a automatic updating of it). > > > > Yeah there is a Bodhi ticket about this and I noted that we will need to > > make sure we still work with releng's script if we make the change: > > > > https://github.com/fedora-infra/bodhi/issues/2433 > > Yeah, it sure would be nice if we could just have something run in cron > once a day or so and just update it. Releng has not been good about > running this script often. > > > > We might want to bring up the bigger topic of if we want to bother > > > keeping this. The only current use of it is some rules about going > > > stable in bodhi... are those actually useful? > > > > This sounds like a policy decision, so perhaps the question could be > > posed to FESCo or possibly the packaging committee. It does make some > > intuitive sense to me that we would treat certain packages more > > stringently than we do others, but I don't have data to say one way or > > the other whether the current policies are beneficial. I will say that I > > would feel uncomfortable changing the policy without the blessing of a > > governing body. > > Yes, this is definitely something for FESCo. > They made the update policy that uses it. > > I can take it to them, I just wanted to see if there was a general sense > that it was useful and we should keep it, or it was pointless and we > should drop it. Perhaps I'll post to devel about it to see what the > general feeling is. FWIW I certainly think we should keep it. If anything, with the few folks we currently have who seem to +1 almost any update two seconds after it lands, we might want to make the barrier a bit higher. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/WGSF2SAKYJYBZ6VNBGRQQUY7WIBR2QY3/
Re: Fedora and PDC, road forward
On 06/13/2018 08:04 PM, Kevin Fenzi wrote: > I can take it to them, I just wanted to see if there was a general sense > that it was useful and we should keep it, or it was pointless and we > should drop it. Perhaps I'll post to devel about it to see what the > general feeling is. I only have a slight "intuition" that it seems useful to me, but that's a rather weak statement and I certainly wouldn't argue about it. If you feel more strongly about it, +1 to raising it for discussion! signature.asc Description: OpenPGP digital signature ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/Z4H3D2ZUSFQS4UO5SS3DL34TKOUOMV27/
Re: Fedora and PDC, road forward
On 06/13/2018 03:19 PM, Adam Williamson wrote: > On Wed, 2018-06-13 at 12:03 -0700, Kevin Fenzi wrote: >> On 06/12/2018 07:41 AM, Pierre-Yves Chibon wrote: >>> Good Morning Everyone, >>> >>> So yesterday we held a meeting on #fedora-apps about the future of PDC in >>> Fedora. >>> We kept notes in etherpad at: http://etherpad.osuosl.org/pdc_fedora >>> Here is the gist of it: >>> >>> What do we currently store in PDC: >>> * modules data - the list of what modules have been built, what rpms are in >>> them, >>> and which ones are active or not. >>> * Stream branches, branch ownership, retirement dates (EOL/SLE) >>> * release/life-circle tracking >>>* product and product versions (fedpkg gets active Fedora and EPEL >>> releases >>> from product versions endpont) >> >> As an aside did we ever get that 'in development' addition we wanted so >> we could point everything still using pkgdb to pdc for release status? > > All the code got done but no-one has yet/ever put a process in place to > actually populate Fedora's PDC with the appropriate data, so it's not > there. Things (fedfind, gnome-software...) are still using the > 'collections' JSON for now. Is that the last thing using pkgdb that we know of? It would be pretty sad to keep running pkgdb, and pdc and the new thing all at the same time. :( kevin signature.asc Description: OpenPGP digital signature ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/5ALVVSZ7ZLOWGNSD3V2LETTZMHOCALJG/
Re: Fedora and PDC, road forward
On 06/13/2018 02:50 PM, Randy Barlow wrote: > On 06/13/2018 03:03 PM, Kevin Fenzi wrote: >> ok. Note that this data changes over time, and releng needs a script to >> update it (or better a automatic updating of it). > > Yeah there is a Bodhi ticket about this and I noted that we will need to > make sure we still work with releng's script if we make the change: > > https://github.com/fedora-infra/bodhi/issues/2433 Yeah, it sure would be nice if we could just have something run in cron once a day or so and just update it. Releng has not been good about running this script often. >> We might want to bring up the bigger topic of if we want to bother >> keeping this. The only current use of it is some rules about going >> stable in bodhi... are those actually useful? > > This sounds like a policy decision, so perhaps the question could be > posed to FESCo or possibly the packaging committee. It does make some > intuitive sense to me that we would treat certain packages more > stringently than we do others, but I don't have data to say one way or > the other whether the current policies are beneficial. I will say that I > would feel uncomfortable changing the policy without the blessing of a > governing body. Yes, this is definitely something for FESCo. They made the update policy that uses it. I can take it to them, I just wanted to see if there was a general sense that it was useful and we should keep it, or it was pointless and we should drop it. Perhaps I'll post to devel about it to see what the general feeling is. kevin signature.asc Description: OpenPGP digital signature ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/LHXEKIFPESJEOVMHB4V4OFMCO4H3LRHV/
Re: Fedora and PDC, road forward
On Wed, 2018-06-13 at 12:03 -0700, Kevin Fenzi wrote: > On 06/12/2018 07:41 AM, Pierre-Yves Chibon wrote: > > Good Morning Everyone, > > > > So yesterday we held a meeting on #fedora-apps about the future of PDC in > > Fedora. > > We kept notes in etherpad at: http://etherpad.osuosl.org/pdc_fedora > > Here is the gist of it: > > > > What do we currently store in PDC: > > * modules data - the list of what modules have been built, what rpms are in > > them, > > and which ones are active or not. > > * Stream branches, branch ownership, retirement dates (EOL/SLE) > > * release/life-circle tracking > >* product and product versions (fedpkg gets active Fedora and EPEL > > releases > > from product versions endpont) > > As an aside did we ever get that 'in development' addition we wanted so > we could point everything still using pkgdb to pdc for release status? All the code got done but no-one has yet/ever put a process in place to actually populate Fedora's PDC with the appropriate data, so it's not there. Things (fedfind, gnome-software...) are still using the 'collections' JSON for now. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/NYKFJDHXPP63HKWWUZCTT5ZCZERC5AXD/
Re: Fedora and PDC, road forward
On 06/13/2018 03:03 PM, Kevin Fenzi wrote: > ok. Note that this data changes over time, and releng needs a script to > update it (or better a automatic updating of it). Yeah there is a Bodhi ticket about this and I noted that we will need to make sure we still work with releng's script if we make the change: https://github.com/fedora-infra/bodhi/issues/2433 > We might want to bring up the bigger topic of if we want to bother > keeping this. The only current use of it is some rules about going > stable in bodhi... are those actually useful? This sounds like a policy decision, so perhaps the question could be posed to FESCo or possibly the packaging committee. It does make some intuitive sense to me that we would treat certain packages more stringently than we do others, but I don't have data to say one way or the other whether the current policies are beneficial. I will say that I would feel uncomfortable changing the policy without the blessing of a governing body. signature.asc Description: OpenPGP digital signature ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/UV5EB5BGLQE7KMAFPQ7AOBDAHAZNVBLE/
Re: Fedora and PDC, road forward
On 06/12/2018 07:41 AM, Pierre-Yves Chibon wrote: > Good Morning Everyone, > > So yesterday we held a meeting on #fedora-apps about the future of PDC in > Fedora. > We kept notes in etherpad at: http://etherpad.osuosl.org/pdc_fedora > Here is the gist of it: > > What do we currently store in PDC: > * modules data - the list of what modules have been built, what rpms are in > them, > and which ones are active or not. > * Stream branches, branch ownership, retirement dates (EOL/SLE) > * release/life-circle tracking >* product and product versions (fedpkg gets active Fedora and EPEL > releases > from product versions endpont) As an aside did we ever get that 'in development' addition we wanted so we could point everything still using pkgdb to pdc for release status? > * critpath boolean on packages (bodhi uses this) > * metadata from composes (RPMS/images) > * "dependency" data about which rpms depend on which other rpms and which > containers include other rpms. This is not used by anything and can be > de-prioritized, dropped. (It was originally going to be used in > Freshmaker.) > Endpoints: > * release-component-relationships > * release-components > * List of all packages: fedora-packages uses this list to know what to index. > It sets up the "for loop" from which fedora-packages pulls data from all > our > other systems. >Endpoints: >* global-components > > > After some discussion the decision we reached was: > * The "dependency" data in its current form can be dropped. It's something > we'll > need in the future but the data structure will most likely need to be > adjusted so let's just drop the existing data and worry about this later. > * The critpath boolean can just be imported into bodhi itself, especially > considering that bodhi is the only tool using this flag. ok. Note that this data changes over time, and releng needs a script to update it (or better a automatic updating of it). We might want to bring up the bigger topic of if we want to bother keeping this. The only current use of it is some rules about going stable in bodhi... are those actually useful? > * The modules data is entirely going to move into MBS > > Everything else (ie: composes metadata, release/life-circle tracking, > package/branches information) will need to find a new home. > This new home will be a new django app using the Django Rest Framework (DRF) > in > which we will import the PDC apps as needed (potentially adjusting them where > desired). > > - kellin has agreed to be the project leader for this work but needs a team to > do the actual work. > - pingou has agreed to look for said team > > > If you have any worries, questions or suggestions or if you want to join the > effort, please let us know :) I'm a little worried about Django. True, we have to maintain a version for mailman3, but it's rhel7/python3. Is this new app going to use that? Alternately if we use Fedora, we need to adjust to new Django versions pretty often (one problem we already hit with PDC). Since this is just a simple api, could we do something more simple? Thanks for leading this effort pingou!!! kevin signature.asc Description: OpenPGP digital signature ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/KH6GPSYO6VHL5HENCONYOTZQZOCPJREZ/
Re: Fedora and PDC
On Thu, Jun 07, 2018 at 06:20:43PM +0200, Pierre-Yves Chibon wrote: > Good Morning Everyone, > > So there has been a number of person who expressed in this topic. > The list I have so far is: > - Clime > - bowlofeggs > - cverna > - nirik > - lsedlar > - abompard > - cqi > - Ken Dreyer > - ralphbean > - mboddu > > I propose that we meet up next week on Monday at 14:00 UTC. It seems we all prefer IRC, I propose we use the #fedora-apps room on freenode. We can always fall back to one of the #fedora-meeting room is -apps is too chatty (in other words: don't do PRs in the middle of the meeting :)). Pierre signature.asc Description: PGP signature ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/XEFBYBUB7ZVUWXGV3UOLX42VTO6AFYY4/
Re: Fedora and PDC
On Fri, Jun 8, 2018 at 8:58 AM, Lubomír Sedlář wrote: > Pierre-Yves Chibon píše v Čt 07. 06. 2018 v 18:20 +0200: > > Good Morning Everyone, > > > > So there has been a number of person who expressed in this topic. > > The list I have so far is: > > - Clime > > - bowlofeggs > > - cverna > > - nirik > > - lsedlar > > - abompard > > - cqi > > - Ken Dreyer > > - ralphbean > > - mboddu > > > > I propose that we meet up next week on Monday at 14:00 UTC. > > > > > > What do you prefer, IRC or video (bluejeans)? > > > > > > What do you all think? > Time is ok. I very slightly prefer irc. clime > The time works for me, and I have no preference over irc or bluejeans. > > Lubomír > ___ > infrastructure mailing list -- infrastructure@lists.fedoraproject.org > To unsubscribe send an email to infrastructure-leave@lists. > fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/ > infrastructure@lists.fedoraproject.org/message/ > OP33GEBX2NQPZTREYSPOHPWSJHSEOKA2/ > ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/537ZVZIRLMPS7MZOFOQZSGFBTR3VWRJP/
Re: Fedora and PDC
Pierre-Yves Chibon píše v Čt 07. 06. 2018 v 18:20 +0200: > Good Morning Everyone, > > So there has been a number of person who expressed in this topic. > The list I have so far is: > - Clime > - bowlofeggs > - cverna > - nirik > - lsedlar > - abompard > - cqi > - Ken Dreyer > - ralphbean > - mboddu > > I propose that we meet up next week on Monday at 14:00 UTC. > > > What do you prefer, IRC or video (bluejeans)? > > > What do you all think? The time works for me, and I have no preference over irc or bluejeans. Lubomír ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/OP33GEBX2NQPZTREYSPOHPWSJHSEOKA2/
Re: Fedora and PDC
On 06/08/2018 12:20 AM, Pierre-Yves Chibon wrote: Good Morning Everyone, So there has been a number of person who expressed in this topic. The list I have so far is: - Clime - bowlofeggs - cverna - nirik - lsedlar - abompard - cqi - Ken Dreyer - ralphbean - mboddu I propose that we meet up next week on Monday at 14:00 UTC. What do you prefer, IRC or video (bluejeans)? It's a bit late for me. I prefer IRC that I can read the discussion later. What do you all think? Pierre ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/C4D4RJXQQ32GUWRS6PXV7ILLHC4F72WG/ -- Regards, Chenxiong Qi ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/MV2EZ4MR5HVHUCJHB34NKYGMG77AOLJI/
Re: Fedora and PDC
On Thu, Jun 7, 2018 at 10:20 AM, Pierre-Yves Chibon wrote: > > > I propose that we meet up next week on Monday at 14:00 UTC. Thanks. That works for me. - Ken ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/G6GI5XVXEGQ63IDT36XKJ3POT3OJQ3YC/
Re: Fedora and PDC
On 06/07/2018 09:20 AM, Pierre-Yves Chibon wrote: > Good Morning Everyone, > > So there has been a number of person who expressed in this topic. > The list I have so far is: > - Clime > - bowlofeggs > - cverna > - nirik > - lsedlar > - abompard > - cqi > - Ken Dreyer > - ralphbean > - mboddu > > I propose that we meet up next week on Monday at 14:00 UTC. A bit early for me, but if others can all make it I can read the logs/notes and chime in after that if I have any questions... > What do you prefer, IRC or video (bluejeans)? IRC would be nice as I can read the discussion later... but again if everyone is doing bluejeans thats fine. kevin signature.asc Description: OpenPGP digital signature ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/6UB3AHQAEKYILJHBUQ52IRPXCKX2HOXW/
Re: Fedora and PDC
On Thu, 7 Jun 2018 at 20:34, Randy Barlow wrote: > > On 06/07/2018 12:20 PM, Pierre-Yves Chibon wrote: > > I propose that we meet up next week on Monday at 14:00 UTC. > > Works for me. That works for me too > > > What do you prefer, IRC or video (bluejeans)? > > I have a slight preference for IRC. No preference > > ___ > infrastructure mailing list -- infrastructure@lists.fedoraproject.org > To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/42J73LGGBXD7LWE4Y7LTDEOGUSIVBQYH/ ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/P42R35I4UINA2HH6EBQLOQG7A7I6T45M/
Re: Fedora and PDC
On 06/07/2018 12:20 PM, Pierre-Yves Chibon wrote: > I propose that we meet up next week on Monday at 14:00 UTC. Works for me. > What do you prefer, IRC or video (bluejeans)? I have a slight preference for IRC. signature.asc Description: OpenPGP digital signature ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/42J73LGGBXD7LWE4Y7LTDEOGUSIVBQYH/
Re: Fedora and PDC
Good Morning Everyone, So there has been a number of person who expressed in this topic. The list I have so far is: - Clime - bowlofeggs - cverna - nirik - lsedlar - abompard - cqi - Ken Dreyer - ralphbean - mboddu I propose that we meet up next week on Monday at 14:00 UTC. What do you prefer, IRC or video (bluejeans)? What do you all think? Pierre signature.asc Description: PGP signature ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/C4D4RJXQQ32GUWRS6PXV7ILLHC4F72WG/
Re: Fedora and PDC
On Sat, Apr 21, 2018 at 5:06 PM, Ken Dreyer wrote: > On Fri, Apr 20, 2018 at 11:34 AM, Pierre-Yves Chibon > wrote: >> I propose we start the discussion on the list and plan for a meeting sometime >> late next week to discuss it further with the interested parties (please >> signal >> yourself) > > I am reimplementing a small piece of PDC as well, because the RH Ceph > Storage product uses/used PDC's > v1/releases/{release_id}/rpm-mapping/{package}/ endpoint. For reference, this project to replace PDC's rpm-mappings endpoint is now open-source at https://github.com/ktdreyer/product-listings-manager I'm not sure if Fedora's currently using the rpm-mappings part of PDC, but I figured I'd mention that this is where we're heading with RH product variant rpm mappings. - Ken ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/E2CDUZSDXGWB4YBAMYF33LZCMLU3XGPG/
Re: Fedora and PDC
On Thu, May 17, 2018 at 10:09:21AM +0800, Chenxiong Qi wrote: > Hi, is it too late? I want to help as well. No it's not too late, PDC is currently a little bit on the back fire due to other priorities but we'll need to get back to it (I expect around early June). I will gather all parties who expressed interest in this thread in a meeting so we can move this forward then :) Thanks for your interest! Pierre ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Re: Fedora and PDC
On Thu, 2018-05-17 at 10:09 +0800, Chenxiong Qi wrote: > Hi, is it too late? I want to help as well. > Hello, I just realized the part "is it too late?" could be ambiguous and confuse people. Sorry for my poor English. I just meant I want to help. :) Regards, Chenxiong Qi > On Fri, 2018-04-20 at 19:34 +0200, Pierre-Yves Chibon wrote: > > Good Morning Everyone, > > > > We have been informed thst week at the upstream devs working on the > > product-definition-center (PDC) are moving away from the project > > and > > are going > > to leave it without a maintainer. Since we adopted PDC for a > > variety > > of data > > flows, this puts us in an awkward position. :( > > > > Ralph and I met up on Tuesday to brainstorm the list of things we > > actively use > > PDC for today and to come up with a contingency plan for how to > > handle them. One > > overarching option is for us (fedora-infra) to take ownership of > > the > > PDC > > codebase as a whole. We didn't fully explore this option, figuring > > that the > > codebase is large and contains lots of tables, endpoints, and > > codepaths that we > > didn't use nor which we plan to use. > > > > Instead, below we've got the four things we use PDC for and some > > options for > > what to do with each. > > > > With the exception of /modules/, one common pattern that we like is > > to > > investigate splitting out the "django apps" that make up PDC into > > their own > > projects. We're calling these "pdc-lite", for fun. See more below. > > > > * Modules > > The data in the /modules/ PDC endpoint is *also* in the MBS > > db. Ralph's > > team is going to just use that and stop using pdc anything for > > modules. > > We're going to need to patch pungi, mbs for local builds, and a > > few other > > places. This should be a relatively low-pain transition. > > > > * Stream branches, branch ownership, retirement dates? > > - SLA/EOL are currently stored in PDC. > > - Queried by releng scripts for retirement, fedrepo-req for new > > branches, > > etc.. > > option #1 > > git repo full of yaml file similar to the override repo > > compiled into a single JSON blob > > Single place for all retired packages > > This feels like the lowest tech option. > > git gives us change control for free... but people easily get > > lost in the > > "UX" of navigating a gigantic git repo full of plaintext > > files. > > option #2 > > pagure's DB/API > > pagure knows nothing about branches currently, so that would > > be > > bigger > > lift > > option #3 > > PDC internally is composed of ~20 "django apps" > > https://github.com/product-definition-center/product-definiti > > on > > -center/tree/master/pdc/apps > > We could pick the 2 or 3 that comprise the branches feature, > > copy them > > out, and turn them into their own service: the "branch > > definition center": > > BDC. > > That would be the "pdc-lite" approach mentionned above, ie: > > PDC > > with only > > the "app" of interest > > > > * release/life-circle tracking? > > option #1: > > PDC lite with just that app of interest > > option #2: > > JSON/yaml file on the proxies > > option #3: > > pkgdb-lite > > option #4: > > ??? > > compose tracking? > > impacted: fedfind > > option #1: > > PDC-lite with just that app of interest > > option #2: > > Drop this entirely? > > Adam probably really wants to keep the record of composes. > > option #3: > > ??? > > > > The "pdc-lite" options are attractive, across the board. One thing > > we get from > > this is greater clarity when discussing things formerly in PDC. If > > something is > > in the branch-definition-center, the compose-definition-center, or > > the > > release-definition-center.. you know what you're talking > > about. Today, when > > talking about whether or not something should be or is in "PDC", it > > is easy to > > get confused. > > > > I propose we start the discussion on the list and plan for a > > meeting > > sometime > > late next week to discuss it further with the interested parties > > (please signal > > yourself) > > > > > > What do you think? > > > > Pierre and Ralph > > > > ___ > > infrastructure mailing list -- infrastructure@lists.fedoraproject.o > > rg > > To unsubscribe send an email to infrastructure-leave@lists.fedorapr > > oj > > ect.org ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Re: Fedora and PDC
Hi, is it too late? I want to help as well. On Fri, 2018-04-20 at 19:34 +0200, Pierre-Yves Chibon wrote: > Good Morning Everyone, > > We have been informed thst week at the upstream devs working on the > product-definition-center (PDC) are moving away from the project and > are going > to leave it without a maintainer. Since we adopted PDC for a variety > of data > flows, this puts us in an awkward position. :( > > Ralph and I met up on Tuesday to brainstorm the list of things we > actively use > PDC for today and to come up with a contingency plan for how to > handle them. One > overarching option is for us (fedora-infra) to take ownership of the > PDC > codebase as a whole. We didn't fully explore this option, figuring > that the > codebase is large and contains lots of tables, endpoints, and > codepaths that we > didn't use nor which we plan to use. > > Instead, below we've got the four things we use PDC for and some > options for > what to do with each. > > With the exception of /modules/, one common pattern that we like is > to > investigate splitting out the "django apps" that make up PDC into > their own > projects. We're calling these "pdc-lite", for fun. See more below. > > * Modules > The data in the /modules/ PDC endpoint is *also* in the MBS > db. Ralph's > team is going to just use that and stop using pdc anything for > modules. > We're going to need to patch pungi, mbs for local builds, and a > few other > places. This should be a relatively low-pain transition. > > * Stream branches, branch ownership, retirement dates? > - SLA/EOL are currently stored in PDC. > - Queried by releng scripts for retirement, fedrepo-req for new > branches, > etc.. > option #1 > git repo full of yaml file similar to the override repo > compiled into a single JSON blob > Single place for all retired packages > This feels like the lowest tech option. > git gives us change control for free... but people easily get > lost in the > "UX" of navigating a gigantic git repo full of plaintext files. > option #2 > pagure's DB/API > pagure knows nothing about branches currently, so that would be > bigger > lift > option #3 > PDC internally is composed of ~20 "django apps" > https://github.com/product-definition-center/product-definition > -center/tree/master/pdc/apps > We could pick the 2 or 3 that comprise the branches feature, > copy them > out, and turn them into their own service: the "branch > definition center": > BDC. > That would be the "pdc-lite" approach mentionned above, ie: PDC > with only > the "app" of interest > > * release/life-circle tracking? > option #1: > PDC lite with just that app of interest > option #2: > JSON/yaml file on the proxies > option #3: > pkgdb-lite > option #4: > ??? > compose tracking? > impacted: fedfind > option #1: > PDC-lite with just that app of interest > option #2: > Drop this entirely? > Adam probably really wants to keep the record of composes. > option #3: > ??? > > The "pdc-lite" options are attractive, across the board. One thing > we get from > this is greater clarity when discussing things formerly in PDC. If > something is > in the branch-definition-center, the compose-definition-center, or > the > release-definition-center.. you know what you're talking > about. Today, when > talking about whether or not something should be or is in "PDC", it > is easy to > get confused. > > I propose we start the discussion on the list and plan for a meeting > sometime > late next week to discuss it further with the interested parties > (please signal > yourself) > > > What do you think? > > Pierre and Ralph > > ___ > infrastructure mailing list -- infrastructure@lists.fedoraproject.org > To unsubscribe send an email to infrastructure-leave@lists.fedoraproj > ect.org ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Re: Fedora and PDC
> * I kinda don't like the tree of files in git. It would work, but it seems poor, like we don't know how to use a database. Along with just strange to work with. We could use Git repo of yaml files as the basis but import data upon each commit for each branch into a locally mounted sqlite DB. Commits would be rejected if the import could not be completed. Then setup just a tiny API upon that sqlite DB served by some micro webapp. Access control, versioning and write access would be handled by Git, but user-friendly, fast, read-only data access would be handled by the sqlite + http API pair. The Git backend can also probably also added later. I am writing this because I think it's an interesting option from the technical perspective, not because I would be actually involved with PDC at the moment. I just wanted to share the idea in case somebody finds it applicable. clime On Mon, Apr 23, 2018 at 7:34 PM, Ken Dreyer wrote: > On Sun, Apr 22, 2018 at 2:23 AM, Stanislav Ochotnicky > wrote: > > The difference is that PDC rpm-mappings API endpoint was result of two > > sources: > > * Manual per-rpm mappings (overrides) - this is sort of suitable if you > >have a product with just a couple source packages so it's manageable > >this way (i.e Ceph case) > > * Results of compose metadata import - this is what Fedora/RHEL uses > >because several thousands of source packages are not manageable > >one-by-one by humans manually. > > > > You could still make a system that would create "PRs" for the generated > > files for second case, but then querying the current state will still be > > a bit tricky. I guess... > > Yeah, the fact that we have (at least) two different input and storage > methods there is a lot of complexity. I'm not sure that's a good > design in 2018. > > Regardless, you're right, I'm envisioning that we'd have a tool to > generate the data commits and PRs (or just commit + push directly). > PDC had included its own rudimentary form of version control for > auditing and message bus integration. Git's experience is much richer. > > - Ken > ___ > infrastructure mailing list -- infrastructure@lists.fedoraproject.org > To unsubscribe send an email to infrastructure-leave@lists. > fedoraproject.org > ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Re: Fedora and PDC
On Sun, Apr 22, 2018 at 2:23 AM, Stanislav Ochotnicky wrote: > The difference is that PDC rpm-mappings API endpoint was result of two > sources: > * Manual per-rpm mappings (overrides) - this is sort of suitable if you >have a product with just a couple source packages so it's manageable >this way (i.e Ceph case) > * Results of compose metadata import - this is what Fedora/RHEL uses >because several thousands of source packages are not manageable >one-by-one by humans manually. > > You could still make a system that would create "PRs" for the generated > files for second case, but then querying the current state will still be > a bit tricky. I guess... Yeah, the fact that we have (at least) two different input and storage methods there is a lot of complexity. I'm not sure that's a good design in 2018. Regardless, you're right, I'm envisioning that we'd have a tool to generate the data commits and PRs (or just commit + push directly). PDC had included its own rudimentary form of version control for auditing and message bus integration. Git's experience is much richer. - Ken ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Re: Fedora and PDC
> > The "pdc-lite" options are attractive, across the board. > I know Django and Django-REST-Framework, and I've made a small contribution to PDC a few months ago, so I may be of use if that's the path we choose. Aurélien ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Re: Fedora and PDC
Hello, please include me in the planning too. I happen to know Django and PDC a bit, and want to help. Lubomír Pierre-Yves Chibon píše v Pá 20. 04. 2018 v 19:34 +0200: > Good Morning Everyone, > > We have been informed thst week at the upstream devs working on the > product-definition-center (PDC) are moving away from the project and > are going to leave it without a maintainer. Since we adopted PDC for > a variety of data flows, this puts us in an awkward position. :( > > Ralph and I met up on Tuesday to brainstorm the list of things we > actively use PDC for today and to come up with a contingency plan for > how to handle them. One overarching option is for us (fedora-infra) > to take ownership of the PDC codebase as a whole. We didn't fully > explore this option, figuring that the codebase is large and contains > lots of tables, endpoints, and codepaths that we didn't use nor which > we plan to use. > > Instead, below we've got the four things we use PDC for and some > options for what to do with each. > > With the exception of /modules/, one common pattern that we like is > to investigate splitting out the "django apps" that make up PDC into > their own projects. We're calling these "pdc-lite", for fun. See > more below. > > * Modules > The data in the /modules/ PDC endpoint is *also* in the MBS > db. Ralph's team is going to just use that and stop using pdc > anything for modules. > We're going to need to patch pungi, mbs for local builds, and a > few other places. This should be a relatively low-pain transition. > > * Stream branches, branch ownership, retirement dates? > - SLA/EOL are currently stored in PDC. > - Queried by releng scripts for retirement, fedrepo-req for new > branches, etc.. > option #1 > git repo full of yaml file similar to the override repo > compiled into a single JSON blob > Single place for all retired packages > This feels like the lowest tech option. > git gives us change control for free... but people easily get > lost in the > "UX" of navigating a gigantic git repo full of plaintext files. > option #2 > pagure's DB/API > pagure knows nothing about branches currently, so that would be > bigger > lift > option #3 > PDC internally is composed of ~20 "django apps" > https://github.com/product-definition-center/product-definition > -center/tree/master/pdc/apps > We could pick the 2 or 3 that comprise the branches feature, > copy them > out, and turn them into their own service: the "branch > definition center": > BDC. > That would be the "pdc-lite" approach mentionned above, ie: PDC > with only > the "app" of interest > > * release/life-circle tracking? > option #1: > PDC lite with just that app of interest > option #2: > JSON/yaml file on the proxies > option #3: > pkgdb-lite > option #4: > ??? > compose tracking? > impacted: fedfind > option #1: > PDC-lite with just that app of interest > option #2: > Drop this entirely? > Adam probably really wants to keep the record of composes. > option #3: > ??? > > The "pdc-lite" options are attractive, across the board. One thing > we get from this is greater clarity when discussing things formerly > in PDC. If something is in the branch-definition-center, the > compose-definition-center, or the release-definition-center.. you > know what you're talking about. Today, when talking about whether or > not something should be or is in "PDC", it is easy to get confused. > > I propose we start the discussion on the list and plan for a meeting > sometime late next week to discuss it further with the interested > parties (please signal yourself) > > > What do you think? > > Pierre and Ralph > > ___ > infrastructure mailing list -- infrastructure@lists.fedoraproject.org > To unsubscribe send an email to infrastructure-leave@lists.fedoraproj > ect.org ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Re: Fedora and PDC
On Sat 21 Apr 2018 05:06:32 PM CEST Ken Dreyer wrote: > On Fri, Apr 20, 2018 at 11:34 AM, Pierre-Yves Chibon > wrote: >> I propose we start the discussion on the list and plan for a meeting sometime >> late next week to discuss it further with the interested parties (please >> signal >> yourself) > > I am reimplementing a small piece of PDC as well, because the RH Ceph > Storage product uses/used PDC's > v1/releases/{release_id}/rpm-mapping/{package}/ endpoint. I'd be > interested in hearing where Fedora may be heading with their > replacement ideas. Please let me know the time of the meeting :) > > Prior to PDC, we had/have another internal solution that used a > PostgreSQL database for what PDC calls rpm-mappings. We thought PDC > would eventually replace this database. With this sytem, it was > impossible to submit changes to the data store unless you were one of > the few people who had rights to submit changes in postgres. Even > reading the data out into something a mere mortal can understand was a > chore. > > Obviously there are many orthogonal problems there, and we could solve > them with better APIs, docs, etc and still keep the Postgres backend. > However it does strike me that flat YAML files in Git are an > attractive backing store. It would make make it easier to submit and > approve data changes, because anyone can submit pull requests to the > flat files, and a CI system could test the changes before merging (or > even auto-merge), etc. The difference is that PDC rpm-mappings API endpoint was result of two sources: * Manual per-rpm mappings (overrides) - this is sort of suitable if you have a product with just a couple source packages so it's manageable this way (i.e Ceph case) * Results of compose metadata import - this is what Fedora/RHEL uses because several thousands of source packages are not manageable one-by-one by humans manually. You could still make a system that would create "PRs" for the generated files for second case, but then querying the current state will still be a bit tricky. I guess... -- Stanislav Ochotnicky Software Engineer, PnT DevOps - Brno PGP: F434 2286 27DC 7D9B 2B64 0866 BCBD 752E 7B08 7241 Red Hat Inc. http://www.redhat.com ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Re: Fedora and PDC
On Fri, Apr 20, 2018 at 11:34 AM, Pierre-Yves Chibon wrote: > I propose we start the discussion on the list and plan for a meeting sometime > late next week to discuss it further with the interested parties (please > signal > yourself) I am reimplementing a small piece of PDC as well, because the RH Ceph Storage product uses/used PDC's v1/releases/{release_id}/rpm-mapping/{package}/ endpoint. I'd be interested in hearing where Fedora may be heading with their replacement ideas. Please let me know the time of the meeting :) Prior to PDC, we had/have another internal solution that used a PostgreSQL database for what PDC calls rpm-mappings. We thought PDC would eventually replace this database. With this sytem, it was impossible to submit changes to the data store unless you were one of the few people who had rights to submit changes in postgres. Even reading the data out into something a mere mortal can understand was a chore. Obviously there are many orthogonal problems there, and we could solve them with better APIs, docs, etc and still keep the Postgres backend. However it does strike me that flat YAML files in Git are an attractive backing store. It would make make it easier to submit and approve data changes, because anyone can submit pull requests to the flat files, and a CI system could test the changes before merging (or even auto-merge), etc. - Ken ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Re: Fedora and PDC
On 04/20/2018 10:34 AM, Pierre-Yves Chibon wrote: > Good Morning Everyone, > > We have been informed thst week at the upstream devs working on the > product-definition-center (PDC) are moving away from the project and are going > to leave it without a maintainer. Since we adopted PDC for a variety of data > flows, this puts us in an awkward position. :( ...snip.. > > I propose we start the discussion on the list and plan for a meeting sometime > late next week to discuss it further with the interested parties (please > signal > yourself) > > > What do you think? So, some generic thoughts: * I kinda don't like the tree of files in git. It would work, but it seems poor, like we don't know how to use a database. Along with just strange to work with. * Right now we have (I think) 3 Django apps... mailman3, pdc and ask. Of course the people doing the work should choose, but it seems like with Django there's always a treadmill of keeping it on the new version thats supported, etc. We ran into this with pdc recently where the version upstream was using wouldn't work with rhel or fedora django without work. * If we are going to split out just the things we need, perhaps this would be a good time to try our hand at microservices? ie, small as we can make them, run in openshift, deploy quickly/all the time, etc If we do that we should make sure to gather requirements and only make those things we need to. Definitely include me in on the planning... thanks. kevin signature.asc Description: OpenPGP digital signature ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Re: Fedora and PDC
On 20 April 2018 at 20:15, Randy Barlow wrote: > Bodhi also uses PDC to determine which packages are critpath per release. > > I'm interested in this topic, so please do invite me to discuss next week! > > > ___ > infrastructure mailing list -- infrastructure@lists.fedoraproject.org > To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org > fedora-packages get the list of packages from PDC during the indexing of the xapian database. You can add me in the discussion too. I am not very familiar with django but could we keep the current PDC setup and just disable the django apps we are not using ? That way we would not have to touch the other apps depending on PDC. ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Re: Fedora and PDC
Bodhi also uses PDC to determine which packages are critpath per release. I'm interested in this topic, so please do invite me to discuss next week! signature.asc Description: OpenPGP digital signature ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Re: Fedora and PDC
* Stream branches, branch ownership, retirement dates? - SLA/EOL are currently stored in PDC. - Queried by releng scripts for retirement, fedrepo-req for new branches, etc.. option #1 git repo full of yaml file similar to the override repo compiled into a single JSON blob Single place for all retired packages This feels like the lowest tech option. git gives us change control for free... but people easily get lost in the "UX" of navigating a gigantic git repo full of plaintext files. I am all for this option because it makes everything public and it is consistent with the /tests/ and modules/ namespace approach that has been more recently introduced. I think that option is actually even attractive. Just noting...I am not a core developer in this subject but this comes to mind as a good way to implement the "branch meta-data storage app". clime On Fri, Apr 20, 2018 at 7:34 PM, Pierre-Yves Chibon wrote: > Good Morning Everyone, > > We have been informed thst week at the upstream devs working on the > product-definition-center (PDC) are moving away from the project and are > going > to leave it without a maintainer. Since we adopted PDC for a variety of > data > flows, this puts us in an awkward position. :( > > Ralph and I met up on Tuesday to brainstorm the list of things we actively > use > PDC for today and to come up with a contingency plan for how to handle > them. One > overarching option is for us (fedora-infra) to take ownership of the PDC > codebase as a whole. We didn't fully explore this option, figuring that the > codebase is large and contains lots of tables, endpoints, and codepaths > that we > didn't use nor which we plan to use. > > Instead, below we've got the four things we use PDC for and some options > for > what to do with each. > > With the exception of /modules/, one common pattern that we like is to > investigate splitting out the "django apps" that make up PDC into their own > projects. We're calling these "pdc-lite", for fun. See more below. > > * Modules > The data in the /modules/ PDC endpoint is *also* in the MBS db. > Ralph's > team is going to just use that and stop using pdc anything for modules. > We're going to need to patch pungi, mbs for local builds, and a few > other > places. This should be a relatively low-pain transition. > > * Stream branches, branch ownership, retirement dates? > - SLA/EOL are currently stored in PDC. > - Queried by releng scripts for retirement, fedrepo-req for new > branches, > etc.. > option #1 > git repo full of yaml file similar to the override repo > compiled into a single JSON blob > Single place for all retired packages > This feels like the lowest tech option. > git gives us change control for free... but people easily get lost > in the > "UX" of navigating a gigantic git repo full of plaintext files. > option #2 > pagure's DB/API > pagure knows nothing about branches currently, so that would be > bigger > lift > option #3 > PDC internally is composed of ~20 "django apps" > https://github.com/product-definition-center/product- > definition-center/tree/master/pdc/apps > We could pick the 2 or 3 that comprise the branches feature, copy > them > out, and turn them into their own service: the "branch definition > center": > BDC. > That would be the "pdc-lite" approach mentionned above, ie: PDC with > only > the "app" of interest > > * release/life-circle tracking? > option #1: > PDC lite with just that app of interest > option #2: > JSON/yaml file on the proxies > option #3: > pkgdb-lite > option #4: > ??? > compose tracking? > impacted: fedfind > option #1: > PDC-lite with just that app of interest > option #2: > Drop this entirely? > Adam probably really wants to keep the record of composes. > option #3: > ??? > > The "pdc-lite" options are attractive, across the board. One thing we get > from > this is greater clarity when discussing things formerly in PDC. If > something is > in the branch-definition-center, the compose-definition-center, or the > release-definition-center.. you know what you're talking about. Today, > when > talking about whether or not something should be or is in "PDC", it is > easy to > get confused. > > I propose we start the discussion on the list and plan for a meeting > sometime > late next week to discuss it further with the interested parties (please > signal > yourself) > > > What do you think? > > Pierre and Ralph > > > ___ > infrastructure mailing list -- infrastructure@lists.fedoraproject.org > To unsubscribe send an email to infrastructure-leave@lists. > fedoraproject.org > > ___ infrastructure mailing list -- infrastructure@lists.fedoraprojec