Re: Modularity and packagers [was Re: PkgDB and the ArbitraryBranching Change]

[Lars Seipel]

On Fri, Jun 09, 2017 at 08:50:58AM +0200, Adam Samalik wrote:
> RPMs... Well, if someone has an application on their server that doesn't
> run in a container, there are still RPMs on a traditional system. But would
> you install multiple versions stuff on that single system? Or would other
> things run in containers? And I'm just curious here, because I managed to
> use containers for basically everything I need. What about other people?
> 
> So, not everything will probably be installable as RPMs on the same system
> at the same time. But I see the world is going to containers, and I'm
> thinking if not being to install all RPMs next to each other on a single
> system is still a real problem. Thoughts?

I can totally see the appeal of containers when you have a uniform
cluster infrastructure that is shared by wildly different
deployments/users with wildly different requirements. That's what things
lke k8s do well.

OTOH, you pay for that nice flexibility with a huge increase in
complexity. Why would I want to put up with that if I'm not managing a
lot of deployments or, heck, when it's just my own development computer?

I may still do it for fun or because the benefits are really worth it in
a specific case, but designating it as the default (and maybe only) way
to work seems wrong[1]. Not when all that's actually required is a
different version of some library.

[1] Especially when considering that our industry as a whole has a very
hard time (euphemism!) producing systems that correctly deal with the
complexity that's already there.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: pkgdb 'collections' API is now inaccurate, will soon go away (was Re: PkgDB and the ArbitraryBranching Change)

[Kevin Kofler]

Adam Williamson wrote:
> pkgdb had an API endpoint, 'collections', which was useful as a
> reliable source of information about available Fedora releases and
> their status. It still exists now, until pkgdb is entirely turned off:
> 
> https://admin.fedoraproject.org/pkgdb/api/collections/
> 
> but as pkgdb was made read-only on August 4th, its data is outdated. It
> shows Fedora 24 as 'active' (rather than 'EOL'), and it has not had
> Fedora 27 added (as would usually be the case when it branched).

[snip]

> Of course, the 'natural' response to this would be to rewrite things
> that use the pkgdb collections API to use PDC instead. However, there's
> a problem with that: PDC does not provide sufficient data. Releases in
> PDC can only be marked as "active" or "inactive". It seems EOL releases
> are marked "inactive" and both stable and under-development releases
> are marked "active" - so it is literally not possible to tell from PDC
> whether a release is stable or Branched (e.g. right now, both Fedora 26
> and Fedora 27 are simply 'active').

It's always the same story in Fedora (both on the infrastructure side and in 
the distro itself): working software gets decommissioned before there is a 
complete replacement and before consumers of the APIs are updated to use 
whatever should be the replacement.

And then we are surprised when reviewers call Fedora a "bleeding-edge 
distro" or a "RHEL beta".

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: pkgdb 'collections' API is now inaccurate, will soon go away (was Re: PkgDB and the ArbitraryBranching Change)

[Matthew Miller]

On Fri, Aug 18, 2017 at 09:51:40PM -0400, Matthew Miller wrote:
> On Fri, Aug 18, 2017 at 08:06:22PM -0500, Michael Catanzaro wrote:
> > >I swear we talked about this somewhere before. I can't find the
> > >ticket, though.
> > Possibly https://pagure.io/fedora-workstation/issue/23
> Yes! That was totally it. Thanks.

I filed https://pagure.io/fedora-infrastructure/issue/6267 requesting
an interim fix.

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: pkgdb 'collections' API is now inaccurate, will soon go away (was Re: PkgDB and the ArbitraryBranching Change)

[Matthew Miller]

On Fri, Aug 18, 2017 at 08:06:22PM -0500, Michael Catanzaro wrote:
> >I swear we talked about this somewhere before. I can't find the
> >ticket, though.
> Possibly https://pagure.io/fedora-workstation/issue/23

Yes! That was totally it. Thanks.

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: pkgdb 'collections' API is now inaccurate, will soon go away (was Re: PkgDB and the ArbitraryBranching Change)

[Michael Catanzaro]

On Fri, Aug 18, 2017 at 5:59 PM, Matthew Miller 
 wrote:
I swear we talked about this somewhere before. I can't find the 
ticket,

though.


Possibly https://pagure.io/fedora-workstation/issue/23
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: pkgdb 'collections' API is now inaccurate, will soon go away (was Re: PkgDB and the ArbitraryBranching Change)

[Matthew Miller]

On Fri, Aug 18, 2017 at 02:13:17PM -0700, Adam Williamson wrote:
> There's an important consequence of this that I only realized today.
> 
> pkgdb had an API endpoint, 'collections', which was useful as a
> reliable source of information about available Fedora releases and
> their status. It still exists now, until pkgdb is entirely turned off:
> 
> https://admin.fedoraproject.org/pkgdb/api/collections/
> 
> but as pkgdb was made read-only on August 4th, its data is outdated. It
> shows Fedora 24 as 'active' (rather than 'EOL'), and it has not had
> Fedora 27 added (as would usually be the case when it branched).

I swear we talked about this somewhere before. I can't find the ticket,
though. My suggestion was to simply replace or redirect to a static
page until a replacement can be created. I think it was a pagure ticket
somewhere, but I'm not sure where.

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

pkgdb 'collections' API is now inaccurate, will soon go away (was Re: PkgDB and the ArbitraryBranching Change)

[Adam Williamson]

On Fri, 2017-05-26 at 15:42 -0400, Ralph Bean wrote:
> To make this happen requires significant infrastructure changes.  Our
> proposed plan[4] is to decommission PkgDB entirely and to replace it
> with a combination of PDC[5] and pagure over dist-git.  (Tangentially,
> getting pagure over dist-git to play nicely with PkgDB was a
> challenge.  This route gets us to a pull-request interface for spec
> files quicker.)

There's an important consequence of this that I only realized today.

pkgdb had an API endpoint, 'collections', which was useful as a
reliable source of information about available Fedora releases and
their status. It still exists now, until pkgdb is entirely turned off:

https://admin.fedoraproject.org/pkgdb/api/collections/

but as pkgdb was made read-only on August 4th, its data is outdated. It
shows Fedora 24 as 'active' (rather than 'EOL'), and it has not had
Fedora 27 added (as would usually be the case when it branched).

My 'fedfind' project provides a couple of helper functions for
discovering the current release, all current stable releases etc.,
which were backed by the collections API. I have various things
downstream of fedfind that rely on these helpers, and I know for sure
there is other code that similarly relies on the collections API; the
one I know about for sure is gnome-software, but I'm pretty sure there
are some others too.

Of course, the 'natural' response to this would be to rewrite things
that use the pkgdb collections API to use PDC instead. However, there's
a problem with that: PDC does not provide sufficient data. Releases in
PDC can only be marked as "active" or "inactive". It seems EOL releases
are marked "inactive" and both stable and under-development releases
are marked "active" - so it is literally not possible to tell from PDC
whether a release is stable or Branched (e.g. right now, both Fedora 26
and Fedora 27 are simply 'active').

There is a long-standing bug report about this for PDC, which I've just
updated: 
https://github.com/product-definition-center/product-definition-center/issues/294

Anyway, just wanted to provide a heads-up about this problem for anyone
who's using collections (either directly or via fedfind). For fedfind
purposes, I've cut a new release - 3.6.1 - which simply has the current
'correct' answers hard coded, and this will be provided as an update.
Ralph Bean says he'll try to help get PDC updated to handle this
situation, and then I'll update fedfind again to get the information
from PDC. If you're using the pkgdb data directly, please be aware that
it is no longer accurate and will disappear entirely, whenever pkgdb is
fully disabled. CCing Richard Hughes for the gnome-software case.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Modularity and packagers [was Re: PkgDB and the ArbitraryBranching Change]

[Hedayat Vatankhah]

/*Adam Samalik*/ wrote on Fri, 9 Jun 2017 08:50:58 +0200:



On Thu, Jun 8, 2017 at 10:49 PM, Randy Barlow 
mailto:bowlofe...@fedoraproject.org>> 
wrote:


On Thu, 2017-06-08 at 22:17 +0200, Adam Samalik wrote:
> You add the package and other people start to use it. That's great
> until you need to change the version, but can't, because other
people
> started to use it as a dependency and it would break their stuff.

I recently heard that it will be impossible to install two packages
that depend on different versions of a dependency at the same
time. For
example, if package A depends on C < 2.0 and package B depends on C >=
2.0, it will be impossible to install A and B on the same system. Is
this true? If so, I worry that we will see fracturing in Fedora as
packages drift apart in their dependencies. If not so, I apologize for
the noise ☺


I would say it's "half truth" right now. :-)

There has been no extra work on changing RPMs. If something conflicts 
now, it will keep conflicting. But!






So, not everything will probably be installable as RPMs on the same 
system at the same time. But I see the world is going to containers, 
and I'm thinking if not being to install all RPMs next to each other 
on a single system is still a real problem. Thoughts?



Actually, I think the Modularity work can (potentially) provide a 
solution for installing multiple versions of the same package for RPM 
(assuming that the packages are properly packaged so that different 
versions don't conflict with each other, e.g. two versions of a library 
like boost).
For example, library versioning makes it possible to install multiple 
versions of the same library in a system. However, RPM doesn't allow it 
(except that you use a different package name for them, so they are NOT 
the same package in RPM's view) except for kernel as an exception. IIRC, 
one of the reasons for this is that RPM doesn't know how to handle 
updates of that package. (e.g. there are foo-1 & foo-2 installed, and 
now foo-3 is available. What happen when you try to update foo?). For 
kernel, it is never updated. New versions are just installed.


However, if Modularity becomes a first class citizen for RPM/DNF, it is 
possible to solve this problem too. You know that you should install (if 
possible!) different versions of the same package simultaneously if they 
are required by different modules, and you know that a new version of 
the package in a module should update the previous version of the 
package from that module.


But, well, using containers and things like FlatPack/Snappy, such 
complexity might seem not necessary. But going that way, the Modularity 
itself maybe also can be replaced largely with them too (the only 
missing peace I can think of right now is when you need an Apache 
version which is newer than that of CentOS/EPEL, but older than that of 
not EOL-ed Fedora releases).


Regards,
Hedayat


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Modularity and packagers [was Re: PkgDB and the ArbitraryBranching Change]

[Adam Samalik]

On Thu, Jun 8, 2017 at 10:49 PM, Randy Barlow 
wrote:

> On Thu, 2017-06-08 at 22:17 +0200, Adam Samalik wrote:
> > You add the package and other people start to use it. That's great
> > until you need to change the version, but can't, because other people
> > started to use it as a dependency and it would break their stuff.
>
> I recently heard that it will be impossible to install two packages
> that depend on different versions of a dependency at the same time. For
> example, if package A depends on C < 2.0 and package B depends on C >=
> 2.0, it will be impossible to install A and B on the same system. Is
> this true? If so, I worry that we will see fracturing in Fedora as
> packages drift apart in their dependencies. If not so, I apologize for
> the noise ☺
>

I would say it's "half truth" right now. :-)

There has been no extra work on changing RPMs. If something conflicts now,
it will keep conflicting. But!

My immediate answer would be containers. I like using them to run services,
I even have my developer environments in containers - so I don't pollute my
system. And I even saw people working on "standalone containers" or "system
containers" which are basically containers installed by DNF, started with
systemd, having all the config files on the local filesystem, tracked by
RPM.

I also heard people talk about Flatpaks built out of modules - that might
be a solution for desktop apps.

RPMs... Well, if someone has an application on their server that doesn't
run in a container, there are still RPMs on a traditional system. But would
you install multiple versions stuff on that single system? Or would other
things run in containers? And I'm just curious here, because I managed to
use containers for basically everything I need. What about other people?

So, not everything will probably be installable as RPMs on the same system
at the same time. But I see the world is going to containers, and I'm
thinking if not being to install all RPMs next to each other on a single
system is still a real problem. Thoughts?

___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
>
>


-- 

Adam Šamalík
---
Software Engineer
Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Modularity and packagers [was Re: PkgDB and the ArbitraryBranching Change]

[Randy Barlow]

On Thu, 2017-06-08 at 22:17 +0200, Adam Samalik wrote:
> You add the package and other people start to use it. That's great
> until you need to change the version, but can't, because other people
> started to use it as a dependency and it would break their stuff.

I recently heard that it will be impossible to install two packages
that depend on different versions of a dependency at the same time. For
example, if package A depends on C < 2.0 and package B depends on C >=
2.0, it will be impossible to install A and B on the same system. Is
this true? If so, I worry that we will see fracturing in Fedora as
packages drift apart in their dependencies. If not so, I apologize for
the noise ☺

signature.asc
Description: This is a digitally signed message part
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Modularity and packagers [was Re: PkgDB and the ArbitraryBranching Change]

[Adam Samalik]

On Thu, Jun 8, 2017 at 8:13 PM, Tom Hughes  wrote:

> On 08/06/17 18:54, Matthew Miller wrote:
>
>> On Thu, Jun 08, 2017 at 06:48:27PM +0100, Tom Hughes wrote:
>>
>>> I mean it would probably still be quite daunting for somebody that
>>> did want to get into more detail I guess but I think I wound up
>>> there following through from some of the other stuff about arbitrary
>>> branching and I was mostly just trying to see what changes I (as a
>>> packager largely avoiding modularity) might find it impossible to
>>> avoid and that was just the point at which I decided I wasn't likely
>>> to find anything useful to that goal there and bailed out.
>>>
>>
>> Tom, you work on Node stuff, right?  Do you think any of this might be
>> useful for packaging and maintaining that in Fedora?
>>
>
> Well from what I understand I don't see how it will help with any of the
> problems that we have internally with packaging Node stuff.
>
> I mean assuming you're talking about the node interpreter and node
> libraries (ie modules but I'll avoid that name as it will be confusing
> here) all being part of a Fedora module then I don't see how it helps at
> all because the issue with node packaging is the number of dependencies
> within the node stack and the rate of change of those dependencies.


I can imagine the artificial branching that is coming with modularity could
be beneficial for you. It will give you the possibility of having package
branches with specific end of life and "support or quality level".

You say that you deal with a large number of dependencies that keep
changing. How often these changes happen? What if you could - and this is
just my idea - create a package of lower quality in a specific branch
(which would take less time), and explicitly say so. You could then use the
package just as a dependency of your thing. And when the dependencies
change and the package is no longer needed, you can just dump it. That's
fine because you said it's just for you.

And I don't want to make it sound like the main selling point of modularity
is bad packaging... :-) Another example: You want to add a package to
Fedora that you just need as a dependency. Two bad things can happen:

1/ The package is already there, but in a different version. You can't just
change it.
2/ You add the package and other people start to use it. That's great until
you need to change the version, but can't, because other people started to
use it as a dependency and it would break their stuff.

Modularity with artificial branching will solve both problems.

Is any of this relevant to you?

PS: I also call Python modules libraries... I guess nothing we do is
necessarily final. And that might include naming things.


>
> Tom
>
> --
> Tom Hughes (t...@compton.nu)
> http://compton.nu/
>



-- 

Adam Šamalík
---
Software Engineer
Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Modularity and packagers [was Re: PkgDB and the ArbitraryBranching Change]

[Tom Hughes]

On 08/06/17 18:54, Matthew Miller wrote:

On Thu, Jun 08, 2017 at 06:48:27PM +0100, Tom Hughes wrote:

I mean it would probably still be quite daunting for somebody that
did want to get into more detail I guess but I think I wound up
there following through from some of the other stuff about arbitrary
branching and I was mostly just trying to see what changes I (as a
packager largely avoiding modularity) might find it impossible to
avoid and that was just the point at which I decided I wasn't likely
to find anything useful to that goal there and bailed out.


Tom, you work on Node stuff, right?  Do you think any of this might be
useful for packaging and maintaining that in Fedora?


Well from what I understand I don't see how it will help with any of the 
problems that we have internally with packaging Node stuff.


I mean assuming you're talking about the node interpreter and node 
libraries (ie modules but I'll avoid that name as it will be confusing 
here) all being part of a Fedora module then I don't see how it helps at 
all because the issue with node packaging is the number of dependencies 
within the node stack and the rate of change of those dependencies.


Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Modularity and packagers [was Re: PkgDB and the ArbitraryBranching Change]

[Matthew Miller]

On Thu, Jun 08, 2017 at 06:48:27PM +0100, Tom Hughes wrote:
> I mean it would probably still be quite daunting for somebody that
> did want to get into more detail I guess but I think I wound up
> there following through from some of the other stuff about arbitrary
> branching and I was mostly just trying to see what changes I (as a
> packager largely avoiding modularity) might find it impossible to
> avoid and that was just the point at which I decided I wasn't likely
> to find anything useful to that goal there and bailed out.

Tom, you work on Node stuff, right?  Do you think any of this might be
useful for packaging and maintaining that in Fedora?

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PkgDB and the ArbitraryBranching Change

[Tom Hughes]

On 08/06/17 18:28, Adam Samalik wrote:

On Thu, Jun 8, 2017 at 7:15 PM, Tom Hughes > wrote:

Speaking for myself I came across that the other day and got as far
as clicking through to the documentation page and seeing that just
the contents was about four screens long and gave up at that point
as it's not something I have much interest in anyway.

Something similar happened in regard to the specific issue here when
comment was invited on the arbitrary branching stuff before Fesco
discussed it in that I clicked through to the document and found it
was a long and detailed list of steps the sysadmins would need to
take to roll it out rather than an explanation of what it meant for
end users and gave up at that point on trying to understand what it
meant beyond moving from pkgdb to pagure over dist-git.


This is actually a good feedback. The landing page should explain the 
main concepts and the documentation should give more detail... but maybe 
the landing page is very high-level and the docs too detailed? And there 
is nothing in between?


I'm being a little unfair actually. Having looked at it again I think 
it's more that I looked at the front page and got the general idea and 
then idly clicked through to the docs and decided that was more detail 
than I wanted to read given the overview didn't sound like it was 
something that interested me that much.


I mean it would probably still be quite daunting for somebody that did 
want to get into more detail I guess but I think I wound up there 
following through from some of the other stuff about arbitrary branching 
and I was mostly just trying to see what changes I (as a packager 
largely avoiding modularity) might find it impossible to avoid and that 
was just the point at which I decided I wasn't likely to find anything 
useful to that goal there and bailed out.


Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PkgDB and the ArbitraryBranching Change

[Adam Samalik]

On Thu, Jun 8, 2017 at 7:15 PM, Tom Hughes  wrote:

> On 08/06/17 17:58, Matthew Miller wrote:
>
>> On Thu, Jun 08, 2017 at 10:38:11AM -0500, Michael Cronenworth wrote:
>>
>>> Normally I ignore any Modularity discussion. It doesn't interest me,
>>> and it doesn't affect any projects I work on. It's my own fault that
>>> this change, which does affect me, was not on my radar. I'm not
>>> looking to stop the change but only express my concern about the
>>> lack of marketing and lack of documentation.
>>>
>>
>> Thanks. Communication is hard. Have you seen the stuff at
>> https://docs.pagure.org/modularity/?
>>
>
> Speaking for myself I came across that the other day and got as far as
> clicking through to the documentation page and seeing that just the
> contents was about four screens long and gave up at that point as it's not
> something I have much interest in anyway.
>
> Something similar happened in regard to the specific issue here when
> comment was invited on the arbitrary branching stuff before Fesco discussed
> it in that I clicked through to the document and found it was a long and
> detailed list of steps the sysadmins would need to take to roll it out
> rather than an explanation of what it meant for end users and gave up at
> that point on trying to understand what it meant beyond moving from pkgdb
> to pagure over dist-git.


This is actually a good feedback. The landing page should explain the main
concepts and the documentation should give more detail... but maybe the
landing page is very high-level and the docs too detailed? And there is
nothing in between?

I talked to someone about a different way to write stuff - starting with a
summary at the top and giving more and more details as you follow reading.
But you could theoretically stop reading at almost any point and it should
still make sense. Maybe I could give it a shot and rewrite some of the
pages - or at least give a higher-level, yet still specific and technical,
summary. Hmm.. let me think about it.

Suggestions? Anything I should prioritize?


>
>
> Tom
>
> --
> Tom Hughes (t...@compton.nu)
> http://compton.nu/
>
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
>



-- 

Adam Šamalík
---
Software Engineer
Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PkgDB and the ArbitraryBranching Change

[Tom Hughes]

On 08/06/17 17:58, Matthew Miller wrote:

On Thu, Jun 08, 2017 at 10:38:11AM -0500, Michael Cronenworth wrote:

Normally I ignore any Modularity discussion. It doesn't interest me,
and it doesn't affect any projects I work on. It's my own fault that
this change, which does affect me, was not on my radar. I'm not
looking to stop the change but only express my concern about the
lack of marketing and lack of documentation.


Thanks. Communication is hard. Have you seen the stuff at
https://docs.pagure.org/modularity/?


Speaking for myself I came across that the other day and got as far as 
clicking through to the documentation page and seeing that just the 
contents was about four screens long and gave up at that point as it's 
not something I have much interest in anyway.


Something similar happened in regard to the specific issue here when 
comment was invited on the arbitrary branching stuff before Fesco 
discussed it in that I clicked through to the document and found it was 
a long and detailed list of steps the sysadmins would need to take to 
roll it out rather than an explanation of what it meant for end users 
and gave up at that point on trying to understand what it meant beyond 
moving from pkgdb to pagure over dist-git.


Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PkgDB and the ArbitraryBranching Change

[Matthew Miller]

On Thu, Jun 08, 2017 at 10:38:11AM -0500, Michael Cronenworth wrote:
> Normally I ignore any Modularity discussion. It doesn't interest me,
> and it doesn't affect any projects I work on. It's my own fault that
> this change, which does affect me, was not on my radar. I'm not
> looking to stop the change but only express my concern about the
> lack of marketing and lack of documentation.

Thanks. Communication is hard. Have you seen the stuff at
https://docs.pagure.org/modularity/?

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PkgDB and the ArbitraryBranching Change

[Michael Cronenworth]

On 06/08/2017 10:24 AM, Matthew Miller wrote:

Speaking as someone on both side of doors... this is not something that
was developed in secret at all. It's something that was implied by the
modularity work — which has been very open — and the change "in the
open last month" is all there is to it. There is no puppetmaster behind
the curtain.


I'm not accusing of any puppetry here.

Normally I ignore any Modularity discussion. It doesn't interest me, and it doesn't 
affect any projects I work on. It's my own fault that this change, which does affect 
me, was not on my radar. I'm not looking to stop the change but only express my 
concern about the lack of marketing and lack of documentation.

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PkgDB and the ArbitraryBranching Change

[Matthew Miller]

On Thu, Jun 08, 2017 at 09:42:45AM -0500, Michael Cronenworth wrote:
> This change, which is a pretty radical change, was only brought out
> in the open last month. It's now being shovelled down our throats
> after being behind closed doors for who knows how long. This is a
> dramatic reversal from the previous open, community-driven, changes
> that Fedora normally sees.

Speaking as someone on both side of doors... this is not something that
was developed in secret at all. It's something that was implied by the
modularity work — which has been very open — and the change "in the
open last month" is all there is to it. There is no puppetmaster behind
the curtain. 

There are plenty of times when I _do_ need to do work to drag things
out to be public, transparent, and community-oriented, but this isn't
one of them, and jumping to accusations doesn't help.

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PkgDB and the ArbitraryBranching Change

[Michael Cronenworth]

On 06/08/2017 09:13 AM, Michael J Gruber wrote:

So, PkgDB now comes with a big fat warning saying:

"Attention! PkgDB will be replaced during the week of July 10th, 2017.
Please read the following for migration instructions:
https://fedoraproject.org/wiki/Infrastructure/WhatHappenedToPkgdb";

If I go there I find no "migration instructions" whatsoever, the
pararagraph on "How Is PkgDB's Functionality Being Replaced" is
ridiculously short and points to a "Pagure over Dist-Git (placeholder
link)" which obviously points to nowhere. All these "to be written" in
the timeline for June (!) don't make me overly confident in that agenda.

Saying "Do something or else" without telling us "what" nor "what else"
is a really good way to put off contributors, but no good way to get
everyone in the boat for the move.

Really, I'm all in for weeding out the git branch madness that we have
now in dist-git (too many cherry-picks, insane merge directions) and I
do hope that entanglich branches from releases will help that purpose.

But please, take a step back and think about how and what you
communicate to packagers, that is "infra-users".


+1

This change, which is a pretty radical change, was only brought out in the open last 
month. It's now being shovelled down our throats after being behind closed doors for 
who knows how long. This is a dramatic reversal from the previous open, 
community-driven, changes that Fedora normally sees.


Without full documentation FESCo approved this? Kevin was the only concerned 
member.

I guess I'll be waiting a week, or two, as usual when these type of changes occur, 
before submitting package updates so all of the bugs are ironed out.

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PkgDB and the ArbitraryBranching Change

[Michael J Gruber]

So, PkgDB now comes with a big fat warning saying:

"Attention! PkgDB will be replaced during the week of July 10th, 2017.
Please read the following for migration instructions:
https://fedoraproject.org/wiki/Infrastructure/WhatHappenedToPkgdb";

If I go there I find no "migration instructions" whatsoever, the
pararagraph on "How Is PkgDB's Functionality Being Replaced" is
ridiculously short and points to a "Pagure over Dist-Git (placeholder
link)" which obviously points to nowhere. All these "to be written" in
the timeline for June (!) don't make me overly confident in that agenda.

Saying "Do something or else" without telling us "what" nor "what else"
is a really good way to put off contributors, but no good way to get
everyone in the boat for the move.

Really, I'm all in for weeding out the git branch madness that we have
now in dist-git (too many cherry-picks, insane merge directions) and I
do hope that entanglich branches from releases will help that purpose.

But please, take a step back and think about how and what you
communicate to packagers, that is "infra-users".

Michael

Adam Samalik venit, vidit, dixit 04.06.2017 12:43:
> This enables us to have branches that make more sense for individual
> packages - so we can save work by having just one branch for one version
> acrsoss releases, or to offer more versions or "streams". A slide [1]
> from my recent talk demonstrates the possibilities - and also shows why
> branches are not always just versions. It talks about modules, but it's
> the same for packages, too.
> 
> I like your work, guys! If you want to use any of the graphic from my
> slides in a documentation or anywhere else, please feel free to do so. I
> can even tweak it if you like.
> 
> [1] https://asamalik.fedorapeople.org/modularity-dorscluc-2017/#/1/2
> 
> On Fri, May 26, 2017 at 9:42 PM, Ralph Bean  > wrote:
> 
> Hello,
> 
> As part of the Factory 2.0 and Modularity efforts[1], we’ve been
> developing a plan to migrate to an “arbitrary” branching model from
> our current model of one branch per release (as had been discussed at
> Flock and DevConf[2]).
> 
> The main motivation behind this is to enable functionality required by
> Modularity[3] and to ultimately reduce some package maintenance
> burden. For some packages, it makes sense to have only a single branch
> that feeds into multiple releases. For other packages, it makes sense
> to have multiple branches which correlate with multiple upstream minor
> releases. Today, our source branches are tied to the distro release,
> via PkgDB.  We want to decouple that and use modules to put it all
> back together again.
> 
> To make this happen requires significant infrastructure changes.  Our
> proposed plan[4] is to decommission PkgDB entirely and to replace it
> with a combination of PDC[5] and pagure over dist-git.  (Tangentially,
> getting pagure over dist-git to play nicely with PkgDB was a
> challenge.  This route gets us to a pull-request interface for spec
> files quicker.)
> 
> We have brought this Change to FESCo[6][7][8] who expressed general
> agreement on the project but also concern that the community may be
> caught by off guard by the removal of PkgDB. As part of this change,
> we have proposed a timeline[9] that outlines the steps we plan to take
> to actually proceed with the migration. Please review that if you have
> time and provide feedback. We are most concerned with missing
> scripts/tools that may rely on PkgDB’s API. If you can think of any
> that we may have overlooked, please let us know and we will add it to
> the timeline!
> 
> We are meeting again with FESCo next Friday, June 2nd, where a
> decision will be made on the Change. Any feedback before that would be
> greatly appreciated.
> 
> Ralph and Matt,
> From the so-called Factory 2.0 team
> 
> [1] https://fedoraproject.org/wiki/Infrastructure/Factory2
> 
> [2] https://youtu.be/5gqccjyjwFk?t=26m27s
> 
> [3] https://docs.pagure.org/modularity/
> 
> [4]
> 
> https://fedoraproject.org/wiki/Infrastructure/Factory2/Focus/ArbitraryBranching
> 
> 
> [5] https://fedoraproject.org/wiki/Changes/ProductDefinitionCenter
> 
> [6] https://fedoraproject.org/wiki/Changes/ArbitraryBranching
> 
> [7]
> https://meetbot.fedoraproject.org/teams/fesco/fesco.2017-05-19-16.00.html
> 
> 
> [8]
> https://meetbot.fedoraproject.org/teams/fesco/fesco.2017

Re: PkgDB and the ArbitraryBranching Change

[Adam Samalik]

This enables us to have branches that make more sense for individual
packages - so we can save work by having just one branch for one version
acrsoss releases, or to offer more versions or "streams". A slide [1] from
my recent talk demonstrates the possibilities - and also shows why branches
are not always just versions. It talks about modules, but it's the same for
packages, too.

I like your work, guys! If you want to use any of the graphic from my
slides in a documentation or anywhere else, please feel free to do so. I
can even tweak it if you like.

[1] https://asamalik.fedorapeople.org/modularity-dorscluc-2017/#/1/2

On Fri, May 26, 2017 at 9:42 PM, Ralph Bean  wrote:

> Hello,
>
> As part of the Factory 2.0 and Modularity efforts[1], we’ve been
> developing a plan to migrate to an “arbitrary” branching model from
> our current model of one branch per release (as had been discussed at
> Flock and DevConf[2]).
>
> The main motivation behind this is to enable functionality required by
> Modularity[3] and to ultimately reduce some package maintenance
> burden. For some packages, it makes sense to have only a single branch
> that feeds into multiple releases. For other packages, it makes sense
> to have multiple branches which correlate with multiple upstream minor
> releases. Today, our source branches are tied to the distro release,
> via PkgDB.  We want to decouple that and use modules to put it all
> back together again.
>
> To make this happen requires significant infrastructure changes.  Our
> proposed plan[4] is to decommission PkgDB entirely and to replace it
> with a combination of PDC[5] and pagure over dist-git.  (Tangentially,
> getting pagure over dist-git to play nicely with PkgDB was a
> challenge.  This route gets us to a pull-request interface for spec
> files quicker.)
>
> We have brought this Change to FESCo[6][7][8] who expressed general
> agreement on the project but also concern that the community may be
> caught by off guard by the removal of PkgDB. As part of this change,
> we have proposed a timeline[9] that outlines the steps we plan to take
> to actually proceed with the migration. Please review that if you have
> time and provide feedback. We are most concerned with missing
> scripts/tools that may rely on PkgDB’s API. If you can think of any
> that we may have overlooked, please let us know and we will add it to
> the timeline!
>
> We are meeting again with FESCo next Friday, June 2nd, where a
> decision will be made on the Change. Any feedback before that would be
> greatly appreciated.
>
> Ralph and Matt,
> From the so-called Factory 2.0 team
>
> [1] https://fedoraproject.org/wiki/Infrastructure/Factory2
> [2] https://youtu.be/5gqccjyjwFk?t=26m27s
> [3] https://docs.pagure.org/modularity/
> [4] https://fedoraproject.org/wiki/Infrastructure/Factory2/
> Focus/ArbitraryBranching
> [5] https://fedoraproject.org/wiki/Changes/ProductDefinitionCenter
> [6] https://fedoraproject.org/wiki/Changes/ArbitraryBranching
> [7] https://meetbot.fedoraproject.org/teams/fesco/fesco.2017-05-
> 19-16.00.html
> [8] https://meetbot.fedoraproject.org/teams/fesco/fesco.2017-05-
> 26-16.00.html
> [9] https://fedoraproject.org/wiki/Changes/ArbitraryBranching#Timeline
>
> ___
> devel-announce mailing list -- devel-annou...@lists.fedoraproject.org
> To unsubscribe send an email to devel-announce-leave@lists.
> fedoraproject.org
>
>


-- 

Adam Šamalík
---
Software Engineer
Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PkgDB and the ArbitraryBranching Change

[Mikolaj Izdebski]

On 06/02/2017 06:02 PM, Ralph Bean wrote:
> On Mon, May 29, 2017 at 12:41:57PM +0200, Miroslav Suchý wrote:
>> Dne 26.5.2017 v 21:42 Ralph Bean napsal(a):
>>> Any feedback before that would be
>>> greatly appreciated.
>>
>> PkgDB handles Koschei and upstream monitoring settings too. How I can do 
>> that after the migration?
> 
> The Koschei devs have a feature that will let you manage the koschei
> settings in the koschei web ui.

Correct. Packages can be added through Koschei web UI. This feature was
available since the beginning, but was disabled in Fedora infra.

You can see how this works by looking at Koschei dev instance -
http://209.132.184.96/ - login, click on user name, add packages.

-- 
Mikolaj Izdebski
Software Engineer, Red Hat
IRC: mizdebsk
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PkgDB and the ArbitraryBranching Change

[Matthew Miller]

On Fri, May 26, 2017 at 03:42:25PM -0400, Ralph Bean wrote:
> The main motivation behind this is to enable functionality required by
> Modularity[3] and to ultimately reduce some package maintenance
> burden. For some packages, it makes sense to have only a single branch
> that feeds into multiple releases. For other packages, it makes sense
> to have multiple branches which correlate with multiple upstream minor
> releases. Today, our source branches are tied to the distro release,
> via PkgDB.  We want to decouple that and use modules to put it all
> back together again.

FWIW, I'm super-excited about this change. Thanks for working on it!

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PkgDB and the ArbitraryBranching Change

[Ralph Bean]

On Mon, May 29, 2017 at 12:41:57PM +0200, Miroslav Suchý wrote:
> Dne 26.5.2017 v 21:42 Ralph Bean napsal(a):
> > Any feedback before that would be
> > greatly appreciated.
>
> PkgDB handles Koschei and upstream monitoring settings too. How I can do that 
> after the migration?

The Koschei devs have a feature that will let you manage the koschei
settings in the koschei web ui.

For upstream monitoring (hotness), we're going to extract the current
settings and add them ad a yaml file in dist-git repos for packages
that have this turned on.  We had issues for this in our backlog but
they failed to make it to the timeline in the wiki.  I've added them
just now.

> Does this change somehow affect fedora-packages (aka Moksha) 
> https://apps.fedoraproject.org/packages/ ?

Eh, it does.  That's one we missed -- thanks!

https://github.com/fedora-infra/fedora-packages/blob/develop/fedoracommunity/connectors/pkgdbconnector.py

We'll need to patch that to make requests to the pagure API here too.


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PkgDB and the ArbitraryBranching Change

[Kevin Fenzi]

On 05/29/2017 04:41 AM, Miroslav Suchý wrote:
> Dne 26.5.2017 v 21:42 Ralph Bean napsal(a):
>> Any feedback before that would be
>> greatly appreciated.
> 
> PkgDB handles Koschei and upstream monitoring settings too. How I can do that 
> after the migration?

The wiki page needs updating but the idea was to move these to files in
dist-git. So you just change those files to enable/disable those things
and they look in dist-git for them.

> Does this change somehow affect fedora-packages (aka Moksha) 
> https://apps.fedoraproject.org/packages/ ?

It probibly could yeah, because of it showing the various branches and
versions. It would need to know to look for the new arbitrary branches.

It would also be nice if it could show you the SLA of each branch, etc.

kevin






signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: PkgDB and the ArbitraryBranching Change

[Miroslav Suchý]

Dne 26.5.2017 v 21:42 Ralph Bean napsal(a):
> Any feedback before that would be
> greatly appreciated.

PkgDB handles Koschei and upstream monitoring settings too. How I can do that 
after the migration?

Does this change somehow affect fedora-packages (aka Moksha) 
https://apps.fedoraproject.org/packages/ ?
-- 
Miroslav Suchy, RHCA
Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

PkgDB and the ArbitraryBranching Change

[Ralph Bean]

Hello,

As part of the Factory 2.0 and Modularity efforts[1], we’ve been
developing a plan to migrate to an “arbitrary” branching model from
our current model of one branch per release (as had been discussed at
Flock and DevConf[2]).

The main motivation behind this is to enable functionality required by
Modularity[3] and to ultimately reduce some package maintenance
burden. For some packages, it makes sense to have only a single branch
that feeds into multiple releases. For other packages, it makes sense
to have multiple branches which correlate with multiple upstream minor
releases. Today, our source branches are tied to the distro release,
via PkgDB.  We want to decouple that and use modules to put it all
back together again.

To make this happen requires significant infrastructure changes.  Our
proposed plan[4] is to decommission PkgDB entirely and to replace it
with a combination of PDC[5] and pagure over dist-git.  (Tangentially,
getting pagure over dist-git to play nicely with PkgDB was a
challenge.  This route gets us to a pull-request interface for spec
files quicker.)

We have brought this Change to FESCo[6][7][8] who expressed general
agreement on the project but also concern that the community may be
caught by off guard by the removal of PkgDB. As part of this change,
we have proposed a timeline[9] that outlines the steps we plan to take
to actually proceed with the migration. Please review that if you have
time and provide feedback. We are most concerned with missing
scripts/tools that may rely on PkgDB’s API. If you can think of any
that we may have overlooked, please let us know and we will add it to
the timeline!

We are meeting again with FESCo next Friday, June 2nd, where a
decision will be made on the Change. Any feedback before that would be
greatly appreciated.

Ralph and Matt,
From the so-called Factory 2.0 team

[1] https://fedoraproject.org/wiki/Infrastructure/Factory2
[2] https://youtu.be/5gqccjyjwFk?t=26m27s
[3] https://docs.pagure.org/modularity/
[4] 
https://fedoraproject.org/wiki/Infrastructure/Factory2/Focus/ArbitraryBranching
[5] https://fedoraproject.org/wiki/Changes/ProductDefinitionCenter
[6] https://fedoraproject.org/wiki/Changes/ArbitraryBranching
[7] https://meetbot.fedoraproject.org/teams/fesco/fesco.2017-05-19-16.00.html
[8] https://meetbot.fedoraproject.org/teams/fesco/fesco.2017-05-26-16.00.html
[9] https://fedoraproject.org/wiki/Changes/ArbitraryBranching#Timeline


signature.asc
Description: PGP signature
___
devel-announce mailing list -- devel-annou...@lists.fedoraproject.org
To unsubscribe send an email to devel-announce-le...@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org