Re: Fedora and PDC, road forward

[Clement Verna]

On Tue, 12 Jun 2018 at 16:48, Pierre-Yves Chibon  wrote:
>
> Good Morning Everyone,
>
> So yesterday we held a meeting on #fedora-apps about the future of PDC in 
> Fedora.
> We kept notes in etherpad at: http://etherpad.osuosl.org/pdc_fedora
> Here is the gist of it:
>
> What do we currently store in PDC:
> * modules data - the list of what modules have been built, what rpms are in 
> them,
> and which ones are active or not.
> * Stream branches, branch ownership, retirement dates (EOL/SLE)
> * release/life-circle tracking
>* product and product versions (fedpkg gets active Fedora and EPEL releases
> from product versions endpont)
> * critpath boolean on packages (bodhi uses this)
> * metadata from composes (RPMS/images)
> * "dependency" data about which rpms depend on which other rpms and which
> containers include other rpms.  This is not used by anything and can be
> de-prioritized, dropped.  (It was originally going to be used in 
> Freshmaker.)
>   Endpoints:
>   * release-component-relationships
>   * release-components
> * List of all packages: fedora-packages uses this list to know what to index.
> It sets up the "for loop" from which fedora-packages pulls data from all 
> our
> other systems.
>Endpoints:
>* global-components
>
>
> After some discussion the decision we reached was:
> * The "dependency" data in its current form can be dropped. It's something 
> we'll
> need in the future but the data structure will most likely need to be
> adjusted so let's just drop the existing data and worry about this later.
> * The critpath boolean can just be imported into bodhi itself, especially
> considering that bodhi is the only tool using this flag.

During our last infrastructure meeting [0], tflink mentioned that the
critpath boolean was also used by taskotron. Currently taskotron still
uses pkgdb to get this info, so taskotron will need to be updated to
use the new source of truth for the critpathwe might reconsider moving
this to bodhi and leave in the PDC replacement.

[0] - 
https://meetbot.fedoraproject.org/teams/infrastructure/infrastructure.2018-06-28-14.00.log.html

> * The modules data is entirely going to move into MBS
>
> Everything else (ie: composes metadata, release/life-circle tracking,
> package/branches information) will need to find a new home.
> This new home will be a new django app using the Django Rest Framework (DRF) 
> in
> which we will import the PDC apps as needed (potentially adjusting them where
> desired).
>
> - kellin has agreed to be the project leader for this work but needs a team to
>   do the actual work.
> - pingou has agreed to look for said team
>
>
> If you have any worries, questions or suggestions or if you want to join the
> effort, please let us know :)
>
>
> Finally some links:
> Minutes: 
> https://meetbot.fedoraproject.org/fedora-apps/2018-06-11/pdc_and_fedora.2018-06-11-14.01.html
> Minutes (text): 
> https://meetbot.fedoraproject.org/fedora-apps/2018-06-11/pdc_and_fedora.2018-06-11-14.01.txt
> Log: 
> https://meetbot.fedoraproject.org/fedora-apps/2018-06-11/pdc_and_fedora.2018-06-11-14.01.log.html
>
>
> Have a nice day,
> Pierre
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/N5UHPQE4NZO6O4VZK7WPFWPWVSSN3367/
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/Y2U2Z2PEGUWP72XCPPTI7W66NL7NEYLR/

Re: Fedora and PDC, road forward

[Pierre-Yves Chibon]

On Tue, Jun 19, 2018 at 09:39:25AM +0200, Aurelien Bompard wrote:
> >> https://github.com/noirbizarre/flask-restplus
> 
> Actually, I haven't tried that one. It seems pretty good (from the
> docs), has anybody tried it?

Seen it but not tried it :(


Pierre
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/EDIK5YVHL7IC5GPYTURY3KTN2WAIWWWI/

Re: Fedora and PDC, road forward

[Pierre-Yves Chibon]

On Tue, Jun 19, 2018 at 08:35:54AM +0200, Clement Verna wrote:
> On Tue, 19 Jun 2018 at 01:13, Aurelien Bompard
>  wrote:
> >
> > > Within limits. It should be a version thats supported and gets at least
> > > security updates. Hopefully the one(s) in Fedora follow this.
> >
> > Yeah it's 1.11 now which is LTS, since it'll be the last version to
> > support Python 2
> >
> > > There are a few flask rest frameworks, but I have not much idea how well
> > > they are supported or work.
> > > https://github.com/flask-restful/flask-restful
> > > https://github.com/noirbizarre/flask-restplus
> >
> > Yeah I tried those in my search for something like DRF in the Flask
> > world. They are decent, but far from DRF feature-wise.
> 
> I still think we should keep Django, but I came across this tutorial
> [0] yesterday, and it looks like this is a neat way of dealing with
> REST API using Flask and connexion [1]
> 
> [0] - https://realpython.com/flask-connexion-rest-api/
> [1] - https://github.com/zalando/connexion

The tutorial is quite tempting, it looks quite nice, but the entire auth aspect
isn't covered (though we do have flask-oidc).

Pierre
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/ZP2OZLO3FEK6PJOTHKJU5JGEXFZBEMUJ/

Re: Fedora and PDC, road forward

[Aurelien Bompard]

>> https://github.com/noirbizarre/flask-restplus

Actually, I haven't tried that one. It seems pretty good (from the
docs), has anybody tried it?
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/CYAFYO2OUY4TZKXOPOX2WTUFRIOWGN5O/

Re: Fedora and PDC, road forward

[Clement Verna]

On Tue, 19 Jun 2018 at 01:13, Aurelien Bompard
 wrote:
>
> > Within limits. It should be a version thats supported and gets at least
> > security updates. Hopefully the one(s) in Fedora follow this.
>
> Yeah it's 1.11 now which is LTS, since it'll be the last version to
> support Python 2
>
> > There are a few flask rest frameworks, but I have not much idea how well
> > they are supported or work.
> > https://github.com/flask-restful/flask-restful
> > https://github.com/noirbizarre/flask-restplus
>
> Yeah I tried those in my search for something like DRF in the Flask
> world. They are decent, but far from DRF feature-wise.

I still think we should keep Django, but I came across this tutorial
[0] yesterday, and it looks like this is a neat way of dealing with
REST API using Flask and connexion [1]

[0] - https://realpython.com/flask-connexion-rest-api/
[1] - https://github.com/zalando/connexion

>
> A.
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/N7PGWQ3O252OGT7JLKFWYINAA56AM43P/
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/UH2ROTQIGY4ODNSUARZJF4NRACSG2IDA/

Re: Fedora and PDC, road forward

[Aurelien Bompard]

> Within limits. It should be a version thats supported and gets at least
> security updates. Hopefully the one(s) in Fedora follow this.

Yeah it's 1.11 now which is LTS, since it'll be the last version to
support Python 2

> There are a few flask rest frameworks, but I have not much idea how well
> they are supported or work.
> https://github.com/flask-restful/flask-restful
> https://github.com/noirbizarre/flask-restplus

Yeah I tried those in my search for something like DRF in the Flask
world. They are decent, but far from DRF feature-wise.

A.
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/N7PGWQ3O252OGT7JLKFWYINAA56AM43P/

Re: Fedora and PDC, road forward

[Kevin Fenzi]

On 06/18/2018 01:25 PM, Aurelien Bompard wrote:
>> I'm a little worried about Django. True, we have to maintain a version
>> for mailman3, but it's rhel7/python3. Is this new app going to use that?
> 
> Actually, HyperKitty and Postorius are using Django on Python 2.7. The
> Django version is 1.8 and it's pretty old now.
> I would recommend against starting a new app on Python 2 today and it
> does not look like we have a Python 3 package of Django in EPEL yet.


Yeah, this is the part that worries me... Django 1.8 went out of support
on April 1st (no joke!).

>> Alternately if we use Fedora, we need to adjust to new Django versions
>> pretty often (one problem we already hit with PDC).
> 
> Would it make sense to run it in OpenShift? I'd think so. Then we
> could build it with whichever version works, right?

Within limits. It should be a version thats supported and gets at least
security updates. Hopefully the one(s) in Fedora follow this.

>> Since this is just a simple api, could we do something more simple?
> 
> The thing is that the Django REST Framework library is really
> wonderful and there is no Flask equivalent that I know of. It would
> save us handling of a lot of corner cases, and it has built-in tools
> for versionning the API and thus preserving API compatibility.
> Authentication is also very flexible, etc. It's nice.
> 
> That said, nothing impossible to do in Flask, just longer and possibly
> more error-prone.

Yeah, understood. I'd just like to make sure we have security support
and aren't leaving ourselves exposed. :(

There are a few flask rest frameworks, but I have not much idea how well
they are supported or work.

https://github.com/flask-restful/flask-restful

https://github.com/noirbizarre/flask-restplus

kevin



signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/AQRER4FRC6A277UPGSHVL6VWWTIQYF4C/

Re: Fedora and PDC, road forward

[Aurelien Bompard]

> I'm a little worried about Django. True, we have to maintain a version
> for mailman3, but it's rhel7/python3. Is this new app going to use that?

Actually, HyperKitty and Postorius are using Django on Python 2.7. The
Django version is 1.8 and it's pretty old now.
I would recommend against starting a new app on Python 2 today and it
does not look like we have a Python 3 package of Django in EPEL yet.

> Alternately if we use Fedora, we need to adjust to new Django versions
> pretty often (one problem we already hit with PDC).

Would it make sense to run it in OpenShift? I'd think so. Then we
could build it with whichever version works, right?

> Since this is just a simple api, could we do something more simple?

The thing is that the Django REST Framework library is really
wonderful and there is no Flask equivalent that I know of. It would
save us handling of a lot of corner cases, and it has built-in tools
for versionning the API and thus preserving API compatibility.
Authentication is also very flexible, etc. It's nice.

That said, nothing impossible to do in Flask, just longer and possibly
more error-prone.


Aurélien
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/UO2X3XYQUXXCCB45M5FNR2FSX6HVWPED/

Re: Fedora and PDC, road forward

[Adam Williamson]

On Wed, 2018-06-13 at 17:06 -0700, Kevin Fenzi wrote:
> On 06/13/2018 03:19 PM, Adam Williamson wrote:
> > On Wed, 2018-06-13 at 12:03 -0700, Kevin Fenzi wrote:
> > > On 06/12/2018 07:41 AM, Pierre-Yves Chibon wrote:
> > > > Good Morning Everyone,
> > > > 
> > > > So yesterday we held a meeting on #fedora-apps about the future of PDC 
> > > > in Fedora.
> > > > We kept notes in etherpad at: http://etherpad.osuosl.org/pdc_fedora
> > > > Here is the gist of it:
> > > > 
> > > > What do we currently store in PDC:
> > > > * modules data - the list of what modules have been built, what rpms 
> > > > are in them,
> > > > and which ones are active or not.
> > > > * Stream branches, branch ownership, retirement dates (EOL/SLE)
> > > > * release/life-circle tracking
> > > >* product and product versions (fedpkg gets active Fedora and EPEL 
> > > > releases 
> > > > from product versions endpont) 
> > > 
> > > As an aside did we ever get that 'in development' addition we wanted so
> > > we could point everything still using pkgdb to pdc for release status?
> > 
> > All the code got done but no-one has yet/ever put a process in place to
> > actually populate Fedora's PDC with the appropriate data, so it's not
> > there. Things (fedfind, gnome-software...) are still using the
> > 'collections' JSON for now.
> 
> Is that the last thing using pkgdb that we know of?
> 
> It would be pretty sad to keep running pkgdb, and pdc and the new thing
> all at the same time. :(

It's the last thing *I* know of.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/NZ3DGHONI7TFTX6AS6C2SPDHYRMW5WMB/

Re: Fedora and PDC, road forward

[Adam Williamson]

On Wed, 2018-06-13 at 17:04 -0700, Kevin Fenzi wrote:
> On 06/13/2018 02:50 PM, Randy Barlow wrote:
> > On 06/13/2018 03:03 PM, Kevin Fenzi wrote:
> > > ok. Note that this data changes over time, and releng needs a script to
> > > update it (or better a automatic updating of it).
> > 
> > Yeah there is a Bodhi ticket about this and I noted that we will need to
> > make sure we still work with releng's script if we make the change:
> > 
> > https://github.com/fedora-infra/bodhi/issues/2433
> 
> Yeah, it sure would be nice if we could just have something run in cron
> once a day or so and just update it. Releng has not been good about
> running this script often.
> 
> > > We might want to bring up the bigger topic of if we want to bother
> > > keeping this. The only current use of it is some rules about going
> > > stable in bodhi... are those actually useful?
> > 
> > This sounds like a policy decision, so perhaps the question could be
> > posed to FESCo or possibly the packaging committee. It does make some
> > intuitive sense to me that we would treat certain packages more
> > stringently than we do others, but I don't have data to say one way or
> > the other whether the current policies are beneficial. I will say that I
> > would feel uncomfortable changing the policy without the blessing of a
> > governing body.
> 
> Yes, this is definitely something for FESCo.
> They made the update policy that uses it.
> 
> I can take it to them, I just wanted to see if there was a general sense
> that it was useful and we should keep it, or it was pointless and we
> should drop it. Perhaps I'll post to devel about it to see what the
> general feeling is.

FWIW I certainly think we should keep it. If anything, with the few
folks we currently have who seem to +1 almost any update two seconds
after it lands, we might want to make the barrier a bit higher.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/WGSF2SAKYJYBZ6VNBGRQQUY7WIBR2QY3/

Re: Fedora and PDC, road forward

[Randy Barlow]

On 06/13/2018 08:04 PM, Kevin Fenzi wrote:
> I can take it to them, I just wanted to see if there was a general sense
> that it was useful and we should keep it, or it was pointless and we
> should drop it. Perhaps I'll post to devel about it to see what the
> general feeling is.

I only have a slight "intuition" that it seems useful to me, but that's
a rather weak statement and I certainly wouldn't argue about it. If you
feel more strongly about it, +1 to raising it for discussion!



signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/Z4H3D2ZUSFQS4UO5SS3DL34TKOUOMV27/

Re: Fedora and PDC, road forward

[Kevin Fenzi]

On 06/13/2018 03:19 PM, Adam Williamson wrote:
> On Wed, 2018-06-13 at 12:03 -0700, Kevin Fenzi wrote:
>> On 06/12/2018 07:41 AM, Pierre-Yves Chibon wrote:
>>> Good Morning Everyone,
>>>
>>> So yesterday we held a meeting on #fedora-apps about the future of PDC in 
>>> Fedora.
>>> We kept notes in etherpad at: http://etherpad.osuosl.org/pdc_fedora
>>> Here is the gist of it:
>>>
>>> What do we currently store in PDC:
>>> * modules data - the list of what modules have been built, what rpms are in 
>>> them,
>>> and which ones are active or not.
>>> * Stream branches, branch ownership, retirement dates (EOL/SLE)
>>> * release/life-circle tracking
>>>* product and product versions (fedpkg gets active Fedora and EPEL 
>>> releases 
>>> from product versions endpont) 
>>
>> As an aside did we ever get that 'in development' addition we wanted so
>> we could point everything still using pkgdb to pdc for release status?
> 
> All the code got done but no-one has yet/ever put a process in place to
> actually populate Fedora's PDC with the appropriate data, so it's not
> there. Things (fedfind, gnome-software...) are still using the
> 'collections' JSON for now.

Is that the last thing using pkgdb that we know of?

It would be pretty sad to keep running pkgdb, and pdc and the new thing
all at the same time. :(

kevin




signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/5ALVVSZ7ZLOWGNSD3V2LETTZMHOCALJG/

Re: Fedora and PDC, road forward

[Kevin Fenzi]

On 06/13/2018 02:50 PM, Randy Barlow wrote:
> On 06/13/2018 03:03 PM, Kevin Fenzi wrote:
>> ok. Note that this data changes over time, and releng needs a script to
>> update it (or better a automatic updating of it).
> 
> Yeah there is a Bodhi ticket about this and I noted that we will need to
> make sure we still work with releng's script if we make the change:
> 
> https://github.com/fedora-infra/bodhi/issues/2433

Yeah, it sure would be nice if we could just have something run in cron
once a day or so and just update it. Releng has not been good about
running this script often.

>> We might want to bring up the bigger topic of if we want to bother
>> keeping this. The only current use of it is some rules about going
>> stable in bodhi... are those actually useful?
> 
> This sounds like a policy decision, so perhaps the question could be
> posed to FESCo or possibly the packaging committee. It does make some
> intuitive sense to me that we would treat certain packages more
> stringently than we do others, but I don't have data to say one way or
> the other whether the current policies are beneficial. I will say that I
> would feel uncomfortable changing the policy without the blessing of a
> governing body.

Yes, this is definitely something for FESCo.
They made the update policy that uses it.

I can take it to them, I just wanted to see if there was a general sense
that it was useful and we should keep it, or it was pointless and we
should drop it. Perhaps I'll post to devel about it to see what the
general feeling is.

kevin



signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/LHXEKIFPESJEOVMHB4V4OFMCO4H3LRHV/

Re: Fedora and PDC, road forward

[Adam Williamson]

On Wed, 2018-06-13 at 12:03 -0700, Kevin Fenzi wrote:
> On 06/12/2018 07:41 AM, Pierre-Yves Chibon wrote:
> > Good Morning Everyone,
> > 
> > So yesterday we held a meeting on #fedora-apps about the future of PDC in 
> > Fedora.
> > We kept notes in etherpad at: http://etherpad.osuosl.org/pdc_fedora
> > Here is the gist of it:
> > 
> > What do we currently store in PDC:
> > * modules data - the list of what modules have been built, what rpms are in 
> > them,
> > and which ones are active or not.
> > * Stream branches, branch ownership, retirement dates (EOL/SLE)
> > * release/life-circle tracking
> >* product and product versions (fedpkg gets active Fedora and EPEL 
> > releases 
> > from product versions endpont) 
> 
> As an aside did we ever get that 'in development' addition we wanted so
> we could point everything still using pkgdb to pdc for release status?

All the code got done but no-one has yet/ever put a process in place to
actually populate Fedora's PDC with the appropriate data, so it's not
there. Things (fedfind, gnome-software...) are still using the
'collections' JSON for now.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/NYKFJDHXPP63HKWWUZCTT5ZCZERC5AXD/

Re: Fedora and PDC, road forward

[Randy Barlow]

On 06/13/2018 03:03 PM, Kevin Fenzi wrote:
> ok. Note that this data changes over time, and releng needs a script to
> update it (or better a automatic updating of it).

Yeah there is a Bodhi ticket about this and I noted that we will need to
make sure we still work with releng's script if we make the change:

https://github.com/fedora-infra/bodhi/issues/2433

> We might want to bring up the bigger topic of if we want to bother
> keeping this. The only current use of it is some rules about going
> stable in bodhi... are those actually useful?

This sounds like a policy decision, so perhaps the question could be
posed to FESCo or possibly the packaging committee. It does make some
intuitive sense to me that we would treat certain packages more
stringently than we do others, but I don't have data to say one way or
the other whether the current policies are beneficial. I will say that I
would feel uncomfortable changing the policy without the blessing of a
governing body.



signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/UV5EB5BGLQE7KMAFPQ7AOBDAHAZNVBLE/

Re: Fedora and PDC, road forward

[Kevin Fenzi]

On 06/12/2018 07:41 AM, Pierre-Yves Chibon wrote:
> Good Morning Everyone,
> 
> So yesterday we held a meeting on #fedora-apps about the future of PDC in 
> Fedora.
> We kept notes in etherpad at: http://etherpad.osuosl.org/pdc_fedora
> Here is the gist of it:
> 
> What do we currently store in PDC:
> * modules data - the list of what modules have been built, what rpms are in 
> them,
> and which ones are active or not.
> * Stream branches, branch ownership, retirement dates (EOL/SLE)
> * release/life-circle tracking
>* product and product versions (fedpkg gets active Fedora and EPEL 
> releases 
> from product versions endpont) 

As an aside did we ever get that 'in development' addition we wanted so
we could point everything still using pkgdb to pdc for release status?

> * critpath boolean on packages (bodhi uses this)
> * metadata from composes (RPMS/images)
> * "dependency" data about which rpms depend on which other rpms and which 
> containers include other rpms.  This is not used by anything and can be 
> de-prioritized, dropped.  (It was originally going to be used in 
> Freshmaker.)
>   Endpoints:
>   * release-component-relationships
>   * release-components
> * List of all packages: fedora-packages uses this list to know what to index. 
> It sets up the "for loop" from which fedora-packages pulls data from all 
> our
> other systems.
>Endpoints:
>* global-components
> 
> 
> After some discussion the decision we reached was:
> * The "dependency" data in its current form can be dropped. It's something 
> we'll
> need in the future but the data structure will most likely need to be 
> adjusted so let's just drop the existing data and worry about this later.
> * The critpath boolean can just be imported into bodhi itself, especially 
> considering that bodhi is the only tool using this flag.

ok. Note that this data changes over time, and releng needs a script to
update it (or better a automatic updating of it).

We might want to bring up the bigger topic of if we want to bother
keeping this. The only current use of it is some rules about going
stable in bodhi... are those actually useful?

> * The modules data is entirely going to move into MBS
> 
> Everything else (ie: composes metadata, release/life-circle tracking,
> package/branches information) will need to find a new home.
> This new home will be a new django app using the Django Rest Framework (DRF) 
> in
> which we will import the PDC apps as needed (potentially adjusting them where
> desired).
> 
> - kellin has agreed to be the project leader for this work but needs a team to
>   do the actual work.
> - pingou has agreed to look for said team
> 
> 
> If you have any worries, questions or suggestions or if you want to join the
> effort, please let us know :)

I'm a little worried about Django. True, we have to maintain a version
for mailman3, but it's rhel7/python3. Is this new app going to use that?
Alternately if we use Fedora, we need to adjust to new Django versions
pretty often (one problem we already hit with PDC).

Since this is just a simple api, could we do something more simple?

Thanks for leading this effort pingou!!!

kevin



signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/KH6GPSYO6VHL5HENCONYOTZQZOCPJREZ/

Fedora and PDC, road forward

[Pierre-Yves Chibon]

Good Morning Everyone,

So yesterday we held a meeting on #fedora-apps about the future of PDC in 
Fedora.
We kept notes in etherpad at: http://etherpad.osuosl.org/pdc_fedora
Here is the gist of it:

What do we currently store in PDC:
* modules data - the list of what modules have been built, what rpms are in 
them,
and which ones are active or not.
* Stream branches, branch ownership, retirement dates (EOL/SLE)
* release/life-circle tracking
   * product and product versions (fedpkg gets active Fedora and EPEL releases 
from product versions endpont) 
* critpath boolean on packages (bodhi uses this)
* metadata from composes (RPMS/images)
* "dependency" data about which rpms depend on which other rpms and which 
containers include other rpms.  This is not used by anything and can be 
de-prioritized, dropped.  (It was originally going to be used in 
Freshmaker.)
  Endpoints:
  * release-component-relationships
  * release-components
* List of all packages: fedora-packages uses this list to know what to index. 
It sets up the "for loop" from which fedora-packages pulls data from all our
other systems.
   Endpoints:
   * global-components


After some discussion the decision we reached was:
* The "dependency" data in its current form can be dropped. It's something we'll
need in the future but the data structure will most likely need to be 
adjusted so let's just drop the existing data and worry about this later.
* The critpath boolean can just be imported into bodhi itself, especially 
considering that bodhi is the only tool using this flag.
* The modules data is entirely going to move into MBS

Everything else (ie: composes metadata, release/life-circle tracking,
package/branches information) will need to find a new home.
This new home will be a new django app using the Django Rest Framework (DRF) in
which we will import the PDC apps as needed (potentially adjusting them where
desired).

- kellin has agreed to be the project leader for this work but needs a team to
  do the actual work.
- pingou has agreed to look for said team


If you have any worries, questions or suggestions or if you want to join the
effort, please let us know :)


Finally some links:
Minutes: 
https://meetbot.fedoraproject.org/fedora-apps/2018-06-11/pdc_and_fedora.2018-06-11-14.01.html
  
Minutes (text): 
https://meetbot.fedoraproject.org/fedora-apps/2018-06-11/pdc_and_fedora.2018-06-11-14.01.txt
Log: 
https://meetbot.fedoraproject.org/fedora-apps/2018-06-11/pdc_and_fedora.2018-06-11-14.01.log.html
  


Have a nice day,
Pierre 


signature.asc
Description: PGP signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/N5UHPQE4NZO6O4VZK7WPFWPWVSSN3367/

Re: Fedora and PDC

[Pierre-Yves Chibon]

On Thu, Jun 07, 2018 at 06:20:43PM +0200, Pierre-Yves Chibon wrote:
> Good Morning Everyone,
> 
> So there has been a number of person who expressed in this topic.
> The list I have so far is:
> - Clime
> - bowlofeggs
> - cverna
> - nirik
> - lsedlar
> - abompard
> - cqi
> - Ken Dreyer
> - ralphbean
> - mboddu
> 
> I propose that we meet up next week on Monday at 14:00 UTC. 

It seems we all prefer IRC, I propose we use the #fedora-apps room on freenode.
We can always fall back to one of the #fedora-meeting room is -apps is too
chatty (in other words: don't do PRs in the middle of the meeting :)).


Pierre


signature.asc
Description: PGP signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/XEFBYBUB7ZVUWXGV3UOLX42VTO6AFYY4/

Re: Fedora and PDC

[Michal Novotny]

On Fri, Jun 8, 2018 at 8:58 AM, Lubomír Sedlář 
wrote:

> Pierre-Yves Chibon píše v Čt 07. 06. 2018 v 18:20 +0200:
> > Good Morning Everyone,
> >
> > So there has been a number of person who expressed in this topic.
> > The list I have so far is:
> > - Clime
> > - bowlofeggs
> > - cverna
> > - nirik
> > - lsedlar
> > - abompard
> > - cqi
> > - Ken Dreyer
> > - ralphbean
> > - mboddu
> >
> > I propose that we meet up next week on Monday at 14:00 UTC.
> >
> >
> > What do you prefer, IRC or video (bluejeans)?
> >
> >
> > What do you all think?
>

Time is ok. I very slightly prefer irc.

clime


> The time works for me, and I have no preference over irc or bluejeans.
>
> Lubomír
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-leave@lists.
> fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/
> infrastructure@lists.fedoraproject.org/message/
> OP33GEBX2NQPZTREYSPOHPWSJHSEOKA2/
>
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/537ZVZIRLMPS7MZOFOQZSGFBTR3VWRJP/

Re: Fedora and PDC

[Lubomír Sedlář]

Pierre-Yves Chibon píše v Čt 07. 06. 2018 v 18:20 +0200:
> Good Morning Everyone,
> 
> So there has been a number of person who expressed in this topic.
> The list I have so far is:
> - Clime
> - bowlofeggs
> - cverna
> - nirik
> - lsedlar
> - abompard
> - cqi
> - Ken Dreyer
> - ralphbean
> - mboddu
> 
> I propose that we meet up next week on Monday at 14:00 UTC. 
> 
> 
> What do you prefer, IRC or video (bluejeans)?
> 
> 
> What do you all think?

The time works for me, and I have no preference over irc or bluejeans.

Lubomír
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/OP33GEBX2NQPZTREYSPOHPWSJHSEOKA2/

Re: Fedora and PDC

[Chenxiong Qi]

On 06/08/2018 12:20 AM, Pierre-Yves Chibon wrote:

Good Morning Everyone,

So there has been a number of person who expressed in this topic.
The list I have so far is:
- Clime
- bowlofeggs
- cverna
- nirik
- lsedlar
- abompard
- cqi
- Ken Dreyer
- ralphbean
- mboddu

I propose that we meet up next week on Monday at 14:00 UTC.


What do you prefer, IRC or video (bluejeans)?

It's a bit late for me. I prefer IRC that I can read the discussion later.


What do you all think?

Pierre


___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/C4D4RJXQQ32GUWRS6PXV7ILLHC4F72WG/


--
Regards,
Chenxiong Qi
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/MV2EZ4MR5HVHUCJHB34NKYGMG77AOLJI/

Re: Fedora and PDC

[Ken Dreyer]

On Thu, Jun 7, 2018 at 10:20 AM, Pierre-Yves Chibon  wrote:
> >
> I propose that we meet up next week on Monday at 14:00 UTC.

Thanks. That works for me.

- Ken
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/G6GI5XVXEGQ63IDT36XKJ3POT3OJQ3YC/

Re: Fedora and PDC

[Kevin Fenzi]

On 06/07/2018 09:20 AM, Pierre-Yves Chibon wrote:
> Good Morning Everyone,
> 
> So there has been a number of person who expressed in this topic.
> The list I have so far is:
> - Clime
> - bowlofeggs
> - cverna
> - nirik
> - lsedlar
> - abompard
> - cqi
> - Ken Dreyer
> - ralphbean
> - mboddu
> 
> I propose that we meet up next week on Monday at 14:00 UTC. 

A bit early for me, but if others can all make it I can read the
logs/notes and chime in after that if I have any questions...

> What do you prefer, IRC or video (bluejeans)?

IRC would be nice as I can read the discussion later... but again if
everyone is doing bluejeans thats fine.

kevin



signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/6UB3AHQAEKYILJHBUQ52IRPXCKX2HOXW/

Re: Fedora and PDC

[Clement Verna]

On Thu, 7 Jun 2018 at 20:34, Randy Barlow  wrote:
>
> On 06/07/2018 12:20 PM, Pierre-Yves Chibon wrote:
> > I propose that we meet up next week on Monday at 14:00 UTC.
>
> Works for me.
That works for me too
>
> > What do you prefer, IRC or video (bluejeans)?
>
> I have a slight preference for IRC.
No preference
>
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/42J73LGGBXD7LWE4Y7LTDEOGUSIVBQYH/
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/P42R35I4UINA2HH6EBQLOQG7A7I6T45M/

Re: Fedora and PDC

[Randy Barlow]

On 06/07/2018 12:20 PM, Pierre-Yves Chibon wrote:
> I propose that we meet up next week on Monday at 14:00 UTC.

Works for me.

> What do you prefer, IRC or video (bluejeans)?

I have a slight preference for IRC.



signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/42J73LGGBXD7LWE4Y7LTDEOGUSIVBQYH/

Re: Fedora and PDC

[Pierre-Yves Chibon]

Good Morning Everyone,

So there has been a number of person who expressed in this topic.
The list I have so far is:
- Clime
- bowlofeggs
- cverna
- nirik
- lsedlar
- abompard
- cqi
- Ken Dreyer
- ralphbean
- mboddu

I propose that we meet up next week on Monday at 14:00 UTC. 


What do you prefer, IRC or video (bluejeans)?


What do you all think?

Pierre


signature.asc
Description: PGP signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/C4D4RJXQQ32GUWRS6PXV7ILLHC4F72WG/

Re: Fedora and PDC

[Ken Dreyer]

On Sat, Apr 21, 2018 at 5:06 PM, Ken Dreyer  wrote:
> On Fri, Apr 20, 2018 at 11:34 AM, Pierre-Yves Chibon
>  wrote:
>> I propose we start the discussion on the list and plan for a meeting sometime
>> late next week to discuss it further with the interested parties (please 
>> signal
>> yourself)
>
> I am reimplementing a small piece of PDC as well, because the RH Ceph
> Storage product uses/used PDC's
> v1/releases/{release_id}/rpm-mapping/{package}/ endpoint.

For reference, this project to replace PDC's rpm-mappings endpoint is
now open-source at
https://github.com/ktdreyer/product-listings-manager

I'm not sure if Fedora's currently using the rpm-mappings part of PDC,
but I figured I'd mention that this is where we're heading with RH
product variant rpm mappings.

- Ken
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/E2CDUZSDXGWB4YBAMYF33LZCMLU3XGPG/

Re: Fedora and PDC

[Pierre-Yves Chibon]

On Thu, May 17, 2018 at 10:09:21AM +0800, Chenxiong Qi wrote:
> Hi, is it too late? I want to help as well.

No it's not too late, PDC is currently a little bit on the back fire due to
other priorities but we'll need to get back to it (I expect around early June).

I will gather all parties who expressed interest in this thread in a meeting so
we can move this forward then :)


Thanks for your interest!

Pierre
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Fedora and PDC

[Chenxiong Qi]

On Thu, 2018-05-17 at 10:09 +0800, Chenxiong Qi wrote:
> Hi, is it too late? I want to help as well.
> 

Hello, I just realized the part "is it too late?" could be ambiguous
and confuse people. Sorry for my poor English. I just meant I want to
help. :)

Regards,
Chenxiong Qi

> On Fri, 2018-04-20 at 19:34 +0200, Pierre-Yves Chibon wrote:
> > Good Morning Everyone,
> > 
> > We have been informed thst week at the upstream devs working on the
> > product-definition-center (PDC) are moving away from the project
> > and
> > are going
> > to leave it without a maintainer. Since we adopted PDC for a
> > variety
> > of data
> > flows, this puts us in an awkward position. :(
> > 
> > Ralph and I met up on Tuesday to brainstorm the list of things we
> > actively use
> > PDC for today and to come up with a contingency plan for how to
> > handle them. One
> > overarching option is for us (fedora-infra) to take ownership of
> > the
> > PDC
> > codebase as a whole. We didn't fully explore this option, figuring
> > that the
> > codebase is large and contains lots of tables, endpoints, and
> > codepaths that we
> > didn't use nor which we plan to use.
> > 
> > Instead, below we've got the four things we use PDC for and some
> > options for
> > what to do with each.
> > 
> > With the exception of /modules/, one common pattern that we like is
> > to
> > investigate splitting out the "django apps" that make up PDC into
> > their own
> > projects.  We're calling these "pdc-lite", for fun. See more below.
> > 
> > * Modules
> > The data in the /modules/ PDC endpoint is *also* in the MBS
> > db.  Ralph's
> > team is going to just use that and stop using pdc anything for
> > modules.
> > We're going to need to patch pungi, mbs for local builds, and a
> > few other
> > places.  This should be a relatively low-pain transition.
> > 
> > * Stream branches, branch ownership, retirement dates?
> > - SLA/EOL are currently stored in PDC.
> > - Queried by releng scripts for retirement, fedrepo-req for new
> > branches,
> >   etc..
> > option #1
> >   git repo full of yaml file similar to the override repo
> >   compiled into a single JSON blob
> >   Single place for all retired packages
> >   This feels like the lowest tech option.
> >   git gives us change control for free... but people easily get
> > lost in the
> >   "UX" of navigating a gigantic git repo full of plaintext
> > files.
> > option #2
> >   pagure's DB/API
> >   pagure knows nothing about branches currently, so that would
> > be
> > bigger
> >   lift
> > option #3
> >   PDC internally is composed of ~20 "django apps"
> >   https://github.com/product-definition-center/product-definiti
> > on
> > -center/tree/master/pdc/apps
> >   We could pick the 2 or 3 that comprise the branches feature,
> > copy them
> >   out, and turn them into their own service: the "branch
> > definition center":
> >   BDC.
> >   That would be the "pdc-lite" approach mentionned above, ie:
> > PDC
> > with only
> >   the "app" of interest
> > 
> > * release/life-circle tracking?
> > option #1:
> >   PDC lite with just that app of interest
> > option #2:
> >   JSON/yaml file on the proxies
> > option #3:
> >   pkgdb-lite
> > option #4:
> >   ???
> >   compose tracking?
> > impacted: fedfind
> > option #1:
> >   PDC-lite with just that app of interest
> > option #2:
> >   Drop this entirely?
> >   Adam probably really wants to keep the record of composes.
> > option #3:
> >   ???
> > 
> > The "pdc-lite" options are attractive, across the board.  One thing
> > we get from
> > this is greater clarity when discussing things formerly in PDC.  If
> > something is
> > in the branch-definition-center, the compose-definition-center, or
> > the
> > release-definition-center.. you know what you're talking
> > about.  Today, when
> > talking about whether or not something should be or is in "PDC", it
> > is easy to
> > get confused.
> > 
> > I propose we start the discussion on the list and plan for a
> > meeting
> > sometime
> > late next week to discuss it further with the interested parties
> > (please signal
> > yourself)
> > 
> > 
> > What do you think?
> > 
> > Pierre and Ralph
> > 
> > ___
> > infrastructure mailing list -- infrastructure@lists.fedoraproject.o
> > rg
> > To unsubscribe send an email to infrastructure-leave@lists.fedorapr
> > oj
> > ect.org
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Fedora and PDC

[Chenxiong Qi]

Hi, is it too late? I want to help as well.

On Fri, 2018-04-20 at 19:34 +0200, Pierre-Yves Chibon wrote:
> Good Morning Everyone,
> 
> We have been informed thst week at the upstream devs working on the
> product-definition-center (PDC) are moving away from the project and
> are going
> to leave it without a maintainer. Since we adopted PDC for a variety
> of data
> flows, this puts us in an awkward position. :(
> 
> Ralph and I met up on Tuesday to brainstorm the list of things we
> actively use
> PDC for today and to come up with a contingency plan for how to
> handle them. One
> overarching option is for us (fedora-infra) to take ownership of the
> PDC
> codebase as a whole. We didn't fully explore this option, figuring
> that the
> codebase is large and contains lots of tables, endpoints, and
> codepaths that we
> didn't use nor which we plan to use.
> 
> Instead, below we've got the four things we use PDC for and some
> options for
> what to do with each.
> 
> With the exception of /modules/, one common pattern that we like is
> to
> investigate splitting out the "django apps" that make up PDC into
> their own
> projects.  We're calling these "pdc-lite", for fun. See more below.
> 
> * Modules
> The data in the /modules/ PDC endpoint is *also* in the MBS
> db.  Ralph's
> team is going to just use that and stop using pdc anything for
> modules.
> We're going to need to patch pungi, mbs for local builds, and a
> few other
> places.  This should be a relatively low-pain transition.
> 
> * Stream branches, branch ownership, retirement dates?
> - SLA/EOL are currently stored in PDC.
> - Queried by releng scripts for retirement, fedrepo-req for new
> branches,
>   etc..
> option #1
>   git repo full of yaml file similar to the override repo
>   compiled into a single JSON blob
>   Single place for all retired packages
>   This feels like the lowest tech option.
>   git gives us change control for free... but people easily get
> lost in the
>   "UX" of navigating a gigantic git repo full of plaintext files.
> option #2
>   pagure's DB/API
>   pagure knows nothing about branches currently, so that would be
> bigger
>   lift
> option #3
>   PDC internally is composed of ~20 "django apps"
>   https://github.com/product-definition-center/product-definition
> -center/tree/master/pdc/apps
>   We could pick the 2 or 3 that comprise the branches feature,
> copy them
>   out, and turn them into their own service: the "branch
> definition center":
>   BDC.
>   That would be the "pdc-lite" approach mentionned above, ie: PDC
> with only
>   the "app" of interest
> 
> * release/life-circle tracking?
> option #1:
>   PDC lite with just that app of interest
> option #2:
>   JSON/yaml file on the proxies
> option #3:
>   pkgdb-lite
> option #4:
>   ???
>   compose tracking?
> impacted: fedfind
> option #1:
>   PDC-lite with just that app of interest
> option #2:
>   Drop this entirely?
>   Adam probably really wants to keep the record of composes.
> option #3:
>   ???
> 
> The "pdc-lite" options are attractive, across the board.  One thing
> we get from
> this is greater clarity when discussing things formerly in PDC.  If
> something is
> in the branch-definition-center, the compose-definition-center, or
> the
> release-definition-center.. you know what you're talking
> about.  Today, when
> talking about whether or not something should be or is in "PDC", it
> is easy to
> get confused.
> 
> I propose we start the discussion on the list and plan for a meeting
> sometime
> late next week to discuss it further with the interested parties
> (please signal
> yourself)
> 
> 
> What do you think?
> 
> Pierre and Ralph
> 
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-leave@lists.fedoraproj
> ect.org
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Fedora and PDC

[Michal Novotny]

> * I kinda don't like the tree of files in git. It would work, but it
seems poor, like we don't know how to use a database. Along with just
strange to work with.

We could use Git repo of yaml files as the basis but import data upon
each commit for each branch into a locally mounted sqlite DB. Commits
would be rejected if the import could not be completed. Then setup just
a tiny API upon that sqlite DB served by some micro webapp.

Access control, versioning and write access would be handled by Git,
but user-friendly, fast, read-only data access would be handled by
the sqlite + http API pair.

The Git backend can also probably also added later.

I am writing this because I think it's an interesting option from the
technical perspective, not because I would be actually involved with
PDC at the moment. I just wanted to share the idea in case somebody
finds it applicable.

clime

On Mon, Apr 23, 2018 at 7:34 PM, Ken Dreyer  wrote:

> On Sun, Apr 22, 2018 at 2:23 AM, Stanislav Ochotnicky
>  wrote:
> > The difference is that PDC rpm-mappings API endpoint was result of two
> > sources:
> >  * Manual per-rpm mappings (overrides) - this is sort of suitable if you
> >have a product with just a couple source packages so it's manageable
> >this way (i.e Ceph case)
> >  * Results of compose metadata import - this is what Fedora/RHEL uses
> >because several thousands of source packages are not manageable
> >one-by-one by humans manually.
> >
> > You could still make a system that would create "PRs" for the generated
> > files for second case, but then querying the current state will still be
> > a bit tricky. I guess...
>
> Yeah, the fact that we have (at least) two different input and storage
> methods there is a lot of complexity. I'm not sure that's a good
> design in 2018.
>
> Regardless, you're right, I'm envisioning that we'd have a tool to
> generate the data commits and PRs (or just commit + push directly).
> PDC had included its own rudimentary form of version control for
> auditing and message bus integration. Git's experience is much richer.
>
> - Ken
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-leave@lists.
> fedoraproject.org
>
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Fedora and PDC

[Ken Dreyer]

On Sun, Apr 22, 2018 at 2:23 AM, Stanislav Ochotnicky
 wrote:
> The difference is that PDC rpm-mappings API endpoint was result of two
> sources:
>  * Manual per-rpm mappings (overrides) - this is sort of suitable if you
>have a product with just a couple source packages so it's manageable
>this way (i.e Ceph case)
>  * Results of compose metadata import - this is what Fedora/RHEL uses
>because several thousands of source packages are not manageable
>one-by-one by humans manually.
>
> You could still make a system that would create "PRs" for the generated
> files for second case, but then querying the current state will still be
> a bit tricky. I guess...

Yeah, the fact that we have (at least) two different input and storage
methods there is a lot of complexity. I'm not sure that's a good
design in 2018.

Regardless, you're right, I'm envisioning that we'd have a tool to
generate the data commits and PRs (or just commit + push directly).
PDC had included its own rudimentary form of version control for
auditing and message bus integration. Git's experience is much richer.

- Ken
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Fedora and PDC

[Aurelien Bompard]

> > The "pdc-lite" options are attractive, across the board.
>

I know Django and Django-REST-Framework, and I've made a small contribution
to PDC a few months ago, so I may be of use if that's the path we choose.

Aurélien
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Fedora and PDC

[Lubomír Sedlář]

Hello,
please include me in the planning too. I happen to know Django and PDC
a bit, and want to help.

Lubomír

Pierre-Yves Chibon píše v Pá 20. 04. 2018 v 19:34 +0200:
> Good Morning Everyone,
> 
> We have been informed thst week at the upstream devs working on the
> product-definition-center (PDC) are moving away from the project and
> are going to leave it without a maintainer. Since we adopted PDC for
> a variety of data flows, this puts us in an awkward position. :(
> 
> Ralph and I met up on Tuesday to brainstorm the list of things we
> actively use PDC for today and to come up with a contingency plan for
> how to handle them. One overarching option is for us (fedora-infra)
> to take ownership of the PDC codebase as a whole. We didn't fully
> explore this option, figuring that the codebase is large and contains
> lots of tables, endpoints, and codepaths that we didn't use nor which
> we plan to use.
> 
> Instead, below we've got the four things we use PDC for and some
> options for what to do with each.
> 
> With the exception of /modules/, one common pattern that we like is
> to investigate splitting out the "django apps" that make up PDC into
> their own projects.  We're calling these "pdc-lite", for fun. See
> more below.
> 
> * Modules
> The data in the /modules/ PDC endpoint is *also* in the MBS
> db.  Ralph's team is going to just use that and stop using pdc
> anything for modules.
> We're going to need to patch pungi, mbs for local builds, and a
> few other places.  This should be a relatively low-pain transition.
> 
> * Stream branches, branch ownership, retirement dates?
> - SLA/EOL are currently stored in PDC.
> - Queried by releng scripts for retirement, fedrepo-req for new
> branches, etc..
> option #1
>   git repo full of yaml file similar to the override repo
>   compiled into a single JSON blob
>   Single place for all retired packages
>   This feels like the lowest tech option.
>   git gives us change control for free... but people easily get
> lost in the
>   "UX" of navigating a gigantic git repo full of plaintext files.
> option #2
>   pagure's DB/API
>   pagure knows nothing about branches currently, so that would be
> bigger
>   lift
> option #3
>   PDC internally is composed of ~20 "django apps"
>   https://github.com/product-definition-center/product-definition
> -center/tree/master/pdc/apps
>   We could pick the 2 or 3 that comprise the branches feature,
> copy them
>   out, and turn them into their own service: the "branch
> definition center":
>   BDC.
>   That would be the "pdc-lite" approach mentionned above, ie: PDC
> with only
>   the "app" of interest
> 
> * release/life-circle tracking?
> option #1:
>   PDC lite with just that app of interest
> option #2:
>   JSON/yaml file on the proxies
> option #3:
>   pkgdb-lite
> option #4:
>   ???
>   compose tracking?
> impacted: fedfind
> option #1:
>   PDC-lite with just that app of interest
> option #2:
>   Drop this entirely?
>   Adam probably really wants to keep the record of composes.
> option #3:
>   ???
> 
> The "pdc-lite" options are attractive, across the board.  One thing
> we get from this is greater clarity when discussing things formerly
> in PDC.  If something is in the branch-definition-center, the
> compose-definition-center, or the release-definition-center.. you
> know what you're talking about.  Today, when talking about whether or
> not something should be or is in "PDC", it is easy to get confused.
> 
> I propose we start the discussion on the list and plan for a meeting
> sometime late next week to discuss it further with the interested
> parties (please signal yourself)
> 
> 
> What do you think?
> 
> Pierre and Ralph
> 
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-leave@lists.fedoraproj
> ect.org
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Fedora and PDC

[Ken Dreyer]

On Fri, Apr 20, 2018 at 11:34 AM, Pierre-Yves Chibon
 wrote:
> I propose we start the discussion on the list and plan for a meeting sometime
> late next week to discuss it further with the interested parties (please 
> signal
> yourself)

I am reimplementing a small piece of PDC as well, because the RH Ceph
Storage product uses/used PDC's
v1/releases/{release_id}/rpm-mapping/{package}/ endpoint.  I'd be
interested in hearing where Fedora may be heading with their
replacement ideas. Please let me know the time of the meeting :)

Prior to PDC, we had/have another internal solution that used a
PostgreSQL database for what PDC calls rpm-mappings. We thought PDC
would eventually replace this database. With this sytem, it was
impossible to submit changes to the data store unless you were one of
the few people who had rights to submit changes in postgres. Even
reading the data out into something a mere mortal can understand was a
chore.

Obviously there are many orthogonal problems there, and we could solve
them with better APIs, docs, etc and still keep the Postgres backend.
However it does strike me that flat YAML files in Git are an
attractive backing store. It would make make it easier to submit and
approve data changes, because anyone can submit pull requests to the
flat files, and a CI system could test the changes before merging (or
even auto-merge), etc.

- Ken
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Fedora and PDC

[Kevin Fenzi]

On 04/20/2018 10:34 AM, Pierre-Yves Chibon wrote:
> Good Morning Everyone,
> 
> We have been informed thst week at the upstream devs working on the
> product-definition-center (PDC) are moving away from the project and are going
> to leave it without a maintainer. Since we adopted PDC for a variety of data
> flows, this puts us in an awkward position. :(
...snip..
> 
> I propose we start the discussion on the list and plan for a meeting sometime
> late next week to discuss it further with the interested parties (please 
> signal
> yourself)
> 
> 
> What do you think?

So, some generic thoughts:

* I kinda don't like the tree of files in git. It would work, but it
seems poor, like we don't know how to use a database. Along with just
strange to work with.

* Right now we have (I think) 3 Django apps... mailman3, pdc and ask. Of
course the people doing the work should choose, but it seems like with
Django there's always a treadmill of keeping it on the new version thats
supported, etc. We ran into this with pdc recently where the version
upstream was using wouldn't work with rhel or fedora django without work.

* If we are going to split out just the things we need, perhaps this
would be a good time to try our hand at microservices? ie, small as we
can make them, run in openshift, deploy quickly/all the time, etc If we
do that we should make sure to gather requirements and only make those
things we need to.

Definitely include me in on the planning... thanks.

kevin



signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Fedora and PDC

[Clement Verna]

On 20 April 2018 at 20:15, Randy Barlow  wrote:
> Bodhi also uses PDC to determine which packages are critpath per release.
>
> I'm interested in this topic, so please do invite me to discuss next week!
>
>
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
>

fedora-packages get the list of packages from PDC during the indexing
of the xapian database. You can add me in the discussion too.

I am not very familiar with django but could we keep the current PDC
setup and just disable the django apps we are not using ? That way we
would not have to touch the other apps depending on PDC.
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Fedora and PDC

[Randy Barlow]

Bodhi also uses PDC to determine which packages are critpath per release.

I'm interested in this topic, so please do invite me to discuss next week!



signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Fedora and PDC

[Michal Novotny]

* Stream branches, branch ownership, retirement dates?
- SLA/EOL are currently stored in PDC.
- Queried by releng scripts for retirement, fedrepo-req for new
branches,
  etc..
option #1
  git repo full of yaml file similar to the override repo
  compiled into a single JSON blob
  Single place for all retired packages
  This feels like the lowest tech option.
  git gives us change control for free... but people easily get lost in
the
  "UX" of navigating a gigantic git repo full of plaintext files.

I am all for this option because it makes everything public and it is
consistent
with the /tests/ and modules/ namespace approach that has been more recently
introduced. I think that option is actually even attractive.

Just noting...I am not a core developer in this subject but this comes to
mind
as a good way to implement the "branch meta-data storage app".

clime

On Fri, Apr 20, 2018 at 7:34 PM, Pierre-Yves Chibon 
wrote:

> Good Morning Everyone,
>
> We have been informed thst week at the upstream devs working on the
> product-definition-center (PDC) are moving away from the project and are
> going
> to leave it without a maintainer. Since we adopted PDC for a variety of
> data
> flows, this puts us in an awkward position. :(
>
> Ralph and I met up on Tuesday to brainstorm the list of things we actively
> use
> PDC for today and to come up with a contingency plan for how to handle
> them. One
> overarching option is for us (fedora-infra) to take ownership of the PDC
> codebase as a whole. We didn't fully explore this option, figuring that the
> codebase is large and contains lots of tables, endpoints, and codepaths
> that we
> didn't use nor which we plan to use.
>
> Instead, below we've got the four things we use PDC for and some options
> for
> what to do with each.
>
> With the exception of /modules/, one common pattern that we like is to
> investigate splitting out the "django apps" that make up PDC into their own
> projects.  We're calling these "pdc-lite", for fun. See more below.
>
> * Modules
> The data in the /modules/ PDC endpoint is *also* in the MBS db.
> Ralph's
> team is going to just use that and stop using pdc anything for modules.
> We're going to need to patch pungi, mbs for local builds, and a few
> other
> places.  This should be a relatively low-pain transition.
>
> * Stream branches, branch ownership, retirement dates?
> - SLA/EOL are currently stored in PDC.
> - Queried by releng scripts for retirement, fedrepo-req for new
> branches,
>   etc..
> option #1
>   git repo full of yaml file similar to the override repo
>   compiled into a single JSON blob
>   Single place for all retired packages
>   This feels like the lowest tech option.
>   git gives us change control for free... but people easily get lost
> in the
>   "UX" of navigating a gigantic git repo full of plaintext files.
> option #2
>   pagure's DB/API
>   pagure knows nothing about branches currently, so that would be
> bigger
>   lift
> option #3
>   PDC internally is composed of ~20 "django apps"
>   https://github.com/product-definition-center/product-
> definition-center/tree/master/pdc/apps
>   We could pick the 2 or 3 that comprise the branches feature, copy
> them
>   out, and turn them into their own service: the "branch definition
> center":
>   BDC.
>   That would be the "pdc-lite" approach mentionned above, ie: PDC with
> only
>   the "app" of interest
>
> * release/life-circle tracking?
> option #1:
>   PDC lite with just that app of interest
> option #2:
>   JSON/yaml file on the proxies
> option #3:
>   pkgdb-lite
> option #4:
>   ???
>   compose tracking?
> impacted: fedfind
> option #1:
>   PDC-lite with just that app of interest
> option #2:
>   Drop this entirely?
>   Adam probably really wants to keep the record of composes.
> option #3:
>   ???
>
> The "pdc-lite" options are attractive, across the board.  One thing we get
> from
> this is greater clarity when discussing things formerly in PDC.  If
> something is
> in the branch-definition-center, the compose-definition-center, or the
> release-definition-center.. you know what you're talking about.  Today,
> when
> talking about whether or not something should be or is in "PDC", it is
> easy to
> get confused.
>
> I propose we start the discussion on the list and plan for a meeting
> sometime
> late next week to discuss it further with the interested parties (please
> signal
> yourself)
>
>
> What do you think?
>
> Pierre and Ralph
>
>
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-leave@lists.
> fedoraproject.org
>
>
___
infrastructure mailing list -- 

Fedora and PDC

[Pierre-Yves Chibon]

Good Morning Everyone,

We have been informed thst week at the upstream devs working on the
product-definition-center (PDC) are moving away from the project and are going
to leave it without a maintainer. Since we adopted PDC for a variety of data
flows, this puts us in an awkward position. :(

Ralph and I met up on Tuesday to brainstorm the list of things we actively use
PDC for today and to come up with a contingency plan for how to handle them. One
overarching option is for us (fedora-infra) to take ownership of the PDC
codebase as a whole. We didn't fully explore this option, figuring that the
codebase is large and contains lots of tables, endpoints, and codepaths that we
didn't use nor which we plan to use.

Instead, below we've got the four things we use PDC for and some options for
what to do with each.

With the exception of /modules/, one common pattern that we like is to
investigate splitting out the "django apps" that make up PDC into their own
projects.  We're calling these "pdc-lite", for fun. See more below.

* Modules
The data in the /modules/ PDC endpoint is *also* in the MBS db.  Ralph's
team is going to just use that and stop using pdc anything for modules.
We're going to need to patch pungi, mbs for local builds, and a few other
places.  This should be a relatively low-pain transition.

* Stream branches, branch ownership, retirement dates?
- SLA/EOL are currently stored in PDC.
- Queried by releng scripts for retirement, fedrepo-req for new branches,
  etc..
option #1
  git repo full of yaml file similar to the override repo
  compiled into a single JSON blob
  Single place for all retired packages
  This feels like the lowest tech option.
  git gives us change control for free... but people easily get lost in the
  "UX" of navigating a gigantic git repo full of plaintext files.
option #2
  pagure's DB/API
  pagure knows nothing about branches currently, so that would be bigger
  lift
option #3
  PDC internally is composed of ~20 "django apps"
  
https://github.com/product-definition-center/product-definition-center/tree/master/pdc/apps
  We could pick the 2 or 3 that comprise the branches feature, copy them
  out, and turn them into their own service: the "branch definition center":
  BDC.
  That would be the "pdc-lite" approach mentionned above, ie: PDC with only
  the "app" of interest

* release/life-circle tracking?
option #1:
  PDC lite with just that app of interest
option #2:
  JSON/yaml file on the proxies
option #3:
  pkgdb-lite
option #4:
  ???
  compose tracking?
impacted: fedfind
option #1:
  PDC-lite with just that app of interest
option #2:
  Drop this entirely?
  Adam probably really wants to keep the record of composes.
option #3:
  ???

The "pdc-lite" options are attractive, across the board.  One thing we get from
this is greater clarity when discussing things formerly in PDC.  If something is
in the branch-definition-center, the compose-definition-center, or the
release-definition-center.. you know what you're talking about.  Today, when
talking about whether or not something should be or is in "PDC", it is easy to
get confused.

I propose we start the discussion on the list and plan for a meeting sometime
late next week to discuss it further with the interested parties (please signal
yourself)


What do you think?

Pierre and Ralph



signature.asc
Description: PGP signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org