Re: Modularity vs. libgit

2019-06-26 Thread Petr Pisar
On 2019-06-25, Adam Williamson  wrote:
> So I'm still not confident about the story here. Let's take something
> that does follow semver: it's not going to change API compatibility as
> *often* as libgit2, but ultimately it's gonna happen. And in Rawhide,
> presumably, when a new major version comes out, we want installs to
> move to that new major version; that's how we've always done things in
> the past, that's what Rawhide is for.
>
> So what's the story there? How is that supposed to work?

We had had "master" streams that were deprecated. They lived in "master"
dist-git branch and were supposed to contain the latest available
version regardless of ABI changes. Mainly for integration and
distribution-development purposes. It was meant in the same meaning as
what "Rawhide" means for Fedora releases.

The issue with master streams was that there was no platform:master and
thus they leaked into released distribution. Either when branching
a new a distribution from Rawhide or when using a stream expansion.

But I believe we need something like that. Otherwise how are we supposed
to develop modules in Rawhide? A developer story:

I created perl:5.30 stream that is the latest Perl version now. It
consists of many RPM packages. Some of them will get a new upstream
release between creating perl:5.30 stream and branching F31 from
Rawhide. And some of them will break API. What are we going to do with
these new package releases? Should we ignor them, not integrating them,
until a new perl:5.32 is created next year? Or should we add them into
perl:5.30 and risking the ABI change?

Don't forget I want to provide the stream to other Fedoras. One feasible
way is submit perl:5.30 to testing for released Fedoras, but postpone
pushing to stable until F31 is branched off. That effectively makes
perl:5.30 in-developement until the branching. Is that an accaptable way
of developing modules? The drawback is we align stream life cycle to
Fedora releases.

Another approach would be to build perl:5.30 only for Rawhide
(platform:f31 now) and forbid relengs to tag it into F31 composes on
branching. But this manual process would be error-prone would not scale.
We would need some way of marking a stream as a Rawhide only. And not
only for relengs. There is the issue of modular dependencies. It's not
feasible for packagers to update the modular dependency every time a new
dependency stream emerges. Or is it? Do we need a "master" streams for
building modules?

I hope I missed somthing and there is an obvious and easy way of solving
this issue. Otherwise the modularity is not sustainable for developers.

-- Petr
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-25 Thread Igor Gnatenko
On Tue, Jun 25, 2019 at 8:45 PM Stephen Gallagher 
wrote:

>
>
> On Tue, Jun 25, 2019 at 1:34 PM Adam Williamson <
> adamw...@fedoraproject.org> wrote:
>
>> On Tue, 2019-06-25 at 13:22 -0400, Stephen Gallagher wrote:
>> > On Thu, Jun 20, 2019 at 7:49 PM Adam Williamson <
>> adamw...@fedoraproject.org>
>> > wrote:
>> >
>> > > On Fri, 2019-06-21 at 01:10 +0200, Fabio Valentini wrote:
>> > > > But ... if it's not for different version streams, then what *is it*
>> > > for? 樂
>> > > > (What about the plans to offer different versions of e.g. NodeJS via
>> > > > streams? Is that wrong then, too?)
>> > > >
>> > > > Being able to offer different versions is the only useful use-case
>> I can
>> > > > think of, and if that's not the intended usage ... I don't know why
>> we
>> > > need
>> > > > it, or why somebody should use it at all ...
>> > >
>> > > Eh, I should probably butt out before I get things too far wrong.
>> Let's
>> > > wait for a licensed Modularity Person to show up and explain :)
>> > > Stephen? Someone?
>> > >
>> >
>> > Sorry, I've been on PTO.
>> >
>> > The idea behind a stream is that all updates within that stream are
>> > intended to be fully backwards-compatible. In some cases (e.g. Node.js
>> > major releases) this means that the version number is an appropriate
>> way to
>> > identify the stream. However, that's not always the case. Some projects
>> > follow different versioning schemes and the version number is not
>> actually
>> > representative of the API stability (such as Cockpit, for example).
>> >
>> > So the stream naming is really dependent on what makes sense to the
>> > upstream project. If upstream follows semver.org versioning (like
>> Node.js),
>> > then using the major version number as a stream is exactly right. For a
>> > more complicated example, the IDM (aka FreeIPA) module in RHEL 8 chose
>> to
>> > use a stream called "DL-1" (domain level one). This is because it's a
>> glue
>> > project and a lot of its underlying pieces have differing version
>> numbers,
>> > but the whole stream is designed to support the ABI meeting a
>> specification
>> > they dubbed "DL-1". So any update, regardless of version bump, that
>> retains
>> > that compatibility is free to go into that stream.
>>
>> Well, the libgit streams wouldn't technically violate that, would they?
>> That's sort of the point: they got in trouble by *following* that rule.
>> When a new version came out that was API-incompatible, it showed up in
>> a new stream. If it had been put in an existing stream and all the
>> other streams had been rebuilt, things wouldn't have broken the way
>> they did.
>>
>
> Right, this was information I was missing a week ago. I think libgit2's
> usage of the module streams here is *correct*, except for the part where
> they're trying to change the default in a released Fedora, which is in
> violation of the stable updates policy.
>

We are NOT changing default version of libgit2. We are changing dependency
of some other tools to use new version of libgit2.


>> So I'm still not confident about the story here. Let's take something
>> that does follow semver: it's not going to change API compatibility as
>> *often* as libgit2, but ultimately it's gonna happen. And in Rawhide,
>> presumably, when a new major version comes out, we want installs to
>> move to that new major version; that's how we've always done things in
>> the past, that's what Rawhide is for.
>>
>> So what's the story there? How is that supposed to work?
>>
>
> We have an open RFE for DNF to support recognizing a change in default
> streams and switching over. It is complicated and we're still working out
> the details. Please stay tuned. (Sorry, I can't find the BZ offhand, but
> hopefully someone else can chime in with it).
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-25 Thread Stephen Gallagher
On Tue, Jun 25, 2019 at 1:34 PM Adam Williamson 
wrote:

> On Tue, 2019-06-25 at 13:22 -0400, Stephen Gallagher wrote:
> > On Thu, Jun 20, 2019 at 7:49 PM Adam Williamson <
> adamw...@fedoraproject.org>
> > wrote:
> >
> > > On Fri, 2019-06-21 at 01:10 +0200, Fabio Valentini wrote:
> > > > But ... if it's not for different version streams, then what *is it*
> > > for? 樂
> > > > (What about the plans to offer different versions of e.g. NodeJS via
> > > > streams? Is that wrong then, too?)
> > > >
> > > > Being able to offer different versions is the only useful use-case I
> can
> > > > think of, and if that's not the intended usage ... I don't know why
> we
> > > need
> > > > it, or why somebody should use it at all ...
> > >
> > > Eh, I should probably butt out before I get things too far wrong. Let's
> > > wait for a licensed Modularity Person to show up and explain :)
> > > Stephen? Someone?
> > >
> >
> > Sorry, I've been on PTO.
> >
> > The idea behind a stream is that all updates within that stream are
> > intended to be fully backwards-compatible. In some cases (e.g. Node.js
> > major releases) this means that the version number is an appropriate way
> to
> > identify the stream. However, that's not always the case. Some projects
> > follow different versioning schemes and the version number is not
> actually
> > representative of the API stability (such as Cockpit, for example).
> >
> > So the stream naming is really dependent on what makes sense to the
> > upstream project. If upstream follows semver.org versioning (like
> Node.js),
> > then using the major version number as a stream is exactly right. For a
> > more complicated example, the IDM (aka FreeIPA) module in RHEL 8 chose to
> > use a stream called "DL-1" (domain level one). This is because it's a
> glue
> > project and a lot of its underlying pieces have differing version
> numbers,
> > but the whole stream is designed to support the ABI meeting a
> specification
> > they dubbed "DL-1". So any update, regardless of version bump, that
> retains
> > that compatibility is free to go into that stream.
>
> Well, the libgit streams wouldn't technically violate that, would they?
> That's sort of the point: they got in trouble by *following* that rule.
> When a new version came out that was API-incompatible, it showed up in
> a new stream. If it had been put in an existing stream and all the
> other streams had been rebuilt, things wouldn't have broken the way
> they did.
>

Right, this was information I was missing a week ago. I think libgit2's
usage of the module streams here is *correct*, except for the part where
they're trying to change the default in a released Fedora, which is in
violation of the stable updates policy.


>
> So I'm still not confident about the story here. Let's take something
> that does follow semver: it's not going to change API compatibility as
> *often* as libgit2, but ultimately it's gonna happen. And in Rawhide,
> presumably, when a new major version comes out, we want installs to
> move to that new major version; that's how we've always done things in
> the past, that's what Rawhide is for.
>
> So what's the story there? How is that supposed to work?
>

We have an open RFE for DNF to support recognizing a change in default
streams and switching over. It is complicated and we're still working out
the details. Please stay tuned. (Sorry, I can't find the BZ offhand, but
hopefully someone else can chime in with it).
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-25 Thread Adam Williamson
On Tue, 2019-06-25 at 13:22 -0400, Stephen Gallagher wrote:
> On Thu, Jun 20, 2019 at 7:49 PM Adam Williamson 
> wrote:
> 
> > On Fri, 2019-06-21 at 01:10 +0200, Fabio Valentini wrote:
> > > But ... if it's not for different version streams, then what *is it*
> > for? 樂
> > > (What about the plans to offer different versions of e.g. NodeJS via
> > > streams? Is that wrong then, too?)
> > > 
> > > Being able to offer different versions is the only useful use-case I can
> > > think of, and if that's not the intended usage ... I don't know why we
> > need
> > > it, or why somebody should use it at all ...
> > 
> > Eh, I should probably butt out before I get things too far wrong. Let's
> > wait for a licensed Modularity Person to show up and explain :)
> > Stephen? Someone?
> > 
> 
> Sorry, I've been on PTO.
> 
> The idea behind a stream is that all updates within that stream are
> intended to be fully backwards-compatible. In some cases (e.g. Node.js
> major releases) this means that the version number is an appropriate way to
> identify the stream. However, that's not always the case. Some projects
> follow different versioning schemes and the version number is not actually
> representative of the API stability (such as Cockpit, for example).
> 
> So the stream naming is really dependent on what makes sense to the
> upstream project. If upstream follows semver.org versioning (like Node.js),
> then using the major version number as a stream is exactly right. For a
> more complicated example, the IDM (aka FreeIPA) module in RHEL 8 chose to
> use a stream called "DL-1" (domain level one). This is because it's a glue
> project and a lot of its underlying pieces have differing version numbers,
> but the whole stream is designed to support the ABI meeting a specification
> they dubbed "DL-1". So any update, regardless of version bump, that retains
> that compatibility is free to go into that stream.

Well, the libgit streams wouldn't technically violate that, would they?
That's sort of the point: they got in trouble by *following* that rule.
When a new version came out that was API-incompatible, it showed up in
a new stream. If it had been put in an existing stream and all the
other streams had been rebuilt, things wouldn't have broken the way
they did.

So I'm still not confident about the story here. Let's take something
that does follow semver: it's not going to change API compatibility as
*often* as libgit2, but ultimately it's gonna happen. And in Rawhide,
presumably, when a new major version comes out, we want installs to
move to that new major version; that's how we've always done things in
the past, that's what Rawhide is for.

So what's the story there? How is that supposed to work?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-25 Thread Stephen Gallagher
On Thu, Jun 20, 2019 at 7:49 PM Adam Williamson 
wrote:

> On Fri, 2019-06-21 at 01:10 +0200, Fabio Valentini wrote:
> >
> > But ... if it's not for different version streams, then what *is it*
> for? 樂
> > (What about the plans to offer different versions of e.g. NodeJS via
> > streams? Is that wrong then, too?)
> >
> > Being able to offer different versions is the only useful use-case I can
> > think of, and if that's not the intended usage ... I don't know why we
> need
> > it, or why somebody should use it at all ...
>
> Eh, I should probably butt out before I get things too far wrong. Let's
> wait for a licensed Modularity Person to show up and explain :)
> Stephen? Someone?
>

Sorry, I've been on PTO.

The idea behind a stream is that all updates within that stream are
intended to be fully backwards-compatible. In some cases (e.g. Node.js
major releases) this means that the version number is an appropriate way to
identify the stream. However, that's not always the case. Some projects
follow different versioning schemes and the version number is not actually
representative of the API stability (such as Cockpit, for example).

So the stream naming is really dependent on what makes sense to the
upstream project. If upstream follows semver.org versioning (like Node.js),
then using the major version number as a stream is exactly right. For a
more complicated example, the IDM (aka FreeIPA) module in RHEL 8 chose to
use a stream called "DL-1" (domain level one). This is because it's a glue
project and a lot of its underlying pieces have differing version numbers,
but the whole stream is designed to support the ABI meeting a specification
they dubbed "DL-1". So any update, regardless of version bump, that retains
that compatibility is free to go into that stream.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-21 Thread Adam Samalik
On Fri, Jun 21, 2019 at 5:47 PM Adam Williamson 
wrote:

> On Fri, 2019-06-21 at 13:28 +0200, Adam Samalik wrote:
> > To keep the expectations of Fedora's stable ABI within a release, we
> can't
> > change the default stream of a module mind-release. I know, that's
> probably
> > clear and that's not the issue here. But building on that, at the same
> > time, we can't let "dnf update" to change streams on your system
> > mid-release, because that would be basically breaking the ABI
> expectations
> > as well — different mechanics, same problem.
>
> OK, so, this is one I've been sitting on for a while with this. The
> "within a release" thing keeps coming up here. But did you sufficiently
> consider the fact that Rawhide is "a release", and needs to change like
> this? If your position is "we can't change the default stream of a
> module mid-release", that implies "...but we can change it between
> releases". But how is a 'between releases' change ever going to happen
> if it doesn't happen in Rawhide first...which is technically "mid-
> release"?
>
> AFAICS even if as a matter of policy something shouldn't be done within
> a stable release, anything that can be done between releases needs to
> be *technically* possible 'within a release' (i.e. within Rawhide), or
> else we can't possibly do development properly, no?
>

Very good point!

Yes, we definitely need a mechanism to potentially change streams between
releases. This is to make it the same as when major software versions
change during a distribution upgrade. Even though Modularity respects your
choices of a specific stream, it also need to respect your choices not to
make a choice (if that makes sense). Basically, when you install a package
the way you'd always do by "dnf install package" and it happens to be in a
default module stream, that stream gets automatically enabled and the
package installed. But what happens when there is a new major Fedora
release with a different default stream of that package? Without
Modularity, your package would get upgraded to the new version
automatically when performing the system upgrade. We need to keep the same
behaviour with Modularity as well. So the stream needs to be switched
automatically during the upgrade. A very different story of course would be
you choosing a specific module stream by "dnf module enable module:stream"
and then installing the package from it. In that case, the stream should
not be switched because you've made an explicit choice.

And to be clear, we don't have an existing mechanism for that — but we have
discussed it extensively [0] and I have documented a specific proposal [1]
based on those discussions for that behaviour, discussed that with the DNF
team, and opened a bug [2] based on what we agreed on. However, there was
no progress on it.

To your second point about rawhide, yes, you're absolutely right. The way
we think will be the best to deal with rawhide was to treat it as a special
case, and use a similar mechanism to the one mentioned above to — unless
you chose a specific stream explicitly — keep you on the default stream
with every dnf update, even if it means switching streams. And there are
probably other aspects to take into account before making this a reality.
But again, not much progress there...

[0] https://pagure.io/modularity/issue/108
[1]
https://pagure.io/modularity/working-documents/blob/master/f/lifecycles-upgrades-ownership/upgrades-and-modules.md
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1664427




> --
> Adam Williamson
> Fedora QA Community Monkey
> IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
> http://www.happyassassin.net
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>


-- 

Adam Šamalík
---
Senior Software Engineer
Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-21 Thread Adam Williamson
On Fri, 2019-06-21 at 13:28 +0200, Adam Samalik wrote:
> To keep the expectations of Fedora's stable ABI within a release, we can't
> change the default stream of a module mind-release. I know, that's probably
> clear and that's not the issue here. But building on that, at the same
> time, we can't let "dnf update" to change streams on your system
> mid-release, because that would be basically breaking the ABI expectations
> as well — different mechanics, same problem.

OK, so, this is one I've been sitting on for a while with this. The
"within a release" thing keeps coming up here. But did you sufficiently
consider the fact that Rawhide is "a release", and needs to change like
this? If your position is "we can't change the default stream of a
module mid-release", that implies "...but we can change it between
releases". But how is a 'between releases' change ever going to happen
if it doesn't happen in Rawhide first...which is technically "mid-
release"?

AFAICS even if as a matter of policy something shouldn't be done within
a stable release, anything that can be done between releases needs to
be *technically* possible 'within a release' (i.e. within Rawhide), or
else we can't possibly do development properly, no?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-21 Thread Adam Samalik
On Fri, Jun 21, 2019 at 1:28 PM Adam Samalik  wrote:

>
>
> On Fri, Jun 21, 2019 at 12:08 AM Adam Williamson <
> adamw...@fedoraproject.org> wrote:
>
>> On Thu, 2019-06-20 at 23:48 +0200, Igor Gnatenko wrote:
>> > Hello,
>> >
>> > I just wanted to give you an update from my last discussions on
>> > #fedora-modularity and other places.
>> >
>> > # Problems definition
>> >
>> > * Default modules can't have conflicting dependenices
>> > * Changing dependencies in a stream is not supported
>> >
>> > # Why does libgit2 has to be a module?
>> >
>> > libgit2 is not just one package. It is an ecosystem.
>> >
>> > Right now libgit2 module provides libgit2 itself and python bindings.
>> > While we can obviously provide libgit2_0.26, libgit2_0.27 and such,
>> > this does not help us with python packages. Nobody in sane mind will
>> rename them
>> > and make them conflict (because they are not parallel-installable).
>> >
>> > I wanted to also add ruby bindings to a module, but I never got time
>> > to actually do it.
>> >
>> > # What about dependencies change?
>> >
>> > Let's not lock ourselves into libgit2 story, just take an abstract
>> names.
>> >
>> > Module foo:rolling depends on bar:1. Name of stream "rolling" means to
>> serve
>> > purpose of "user-focused content meant for general use". That means,
>> > if foo's upstream decided to update their bar dependency to a new
>> version,
>> > foo:rolling maintainer would just switch dependency to bar:2.
>>
>> AIUI, the issue here is that *there should not be* bar:1 and bar:2.
>> Module streams are supposed to be 'functional' (as in your 'rolling'
>> example). They are not supposed to be version numbers. This needs to be
>> more clearly explained in the guidelines and enforced, but the way the
>> modularity concept turned out, version-based module streams are *wrong*
>> and should not exist. I recall earlier designs along the way actually
>> sort of envisaged version-based module streams, but that is not how
>> it's supposed to work in the final design.
>>
>
> Versioned module streams should definitely exist, that's one of the main
> points of Modularity. But there are certainly exceptions.
>
> To keep the expectations of Fedora's stable ABI within a release, we can't
> change the default stream of a module mind-release. I know, that's probably
> clear and that's not the issue here. But building on that, at the same
> time, we can't let "dnf update" to change streams on your system
> mid-release, because that would be basically breaking the ABI expectations
> as well — different mechanics, same problem.
>
> So in this specific example — where upstream is changing the ABI very
> often — freezing that package to one major version per Fedora release
> doesn't work. Especially when different packages need a different version
> at the same time. So streams are probably not the right way to deal with
> this specific case. Streams are a great way to provide our users a choice.
> A choice of one of multiple versions of software (mostly leaf applications)
> that are natively built and maintained for their fedora release (as opposed
> to enabling rawhide repos and I don't know what to get a different
> version). But it's not a solution for providing multiple versions of
> libraries that need to be installed at the same time.
>
> When different packages require different version of a library at the same
> time, we can definitely use existing mechanisms for parallel installation
> such as providing different packages with different names, installing the
> library into different places, like compat-packages.
>
> What Modularity could help you with — in an arbitrary case when there is
> bar:1 and bar:2 — is to build your "rolling" module (and other modules)
> against both of those using stream expansion, providing two different
> binaries of the same module stream (with different context), and letting
> DNF to install the right one based on what is on your system, or based on
> defaults. This is one of the new capabilities Modularity brings to the
> table. Of course, that only works if your "rolling" module can be actually
> built against both.
>
> But for cases when your "rolling" module can't be built against both
> versions, existing mechanisms for parallel installation should be used
> instead. Modularity doesn't reimplement those existing mechanisms.
>
> To be fair, I don't think our docs say this in this form anywhere. There
are probably hints that could draw the potential reader to this conclusion,
but we probably need to be more explicit than that — by providing specific
guidance for some model cases, demonstrating where Modularity can help (and
how) and where it can't. I (or anyone) should probably fix that!

>
>
>
>
>> --
>> Adam Williamson
>> Fedora QA Community Monkey
>> IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
>> http://www.happyassassin.net
>> ___
>> devel mailing list -- 

Re: Modularity vs. libgit

2019-06-21 Thread Adam Samalik
On Fri, Jun 21, 2019 at 12:08 AM Adam Williamson 
wrote:

> On Thu, 2019-06-20 at 23:48 +0200, Igor Gnatenko wrote:
> > Hello,
> >
> > I just wanted to give you an update from my last discussions on
> > #fedora-modularity and other places.
> >
> > # Problems definition
> >
> > * Default modules can't have conflicting dependenices
> > * Changing dependencies in a stream is not supported
> >
> > # Why does libgit2 has to be a module?
> >
> > libgit2 is not just one package. It is an ecosystem.
> >
> > Right now libgit2 module provides libgit2 itself and python bindings.
> > While we can obviously provide libgit2_0.26, libgit2_0.27 and such,
> > this does not help us with python packages. Nobody in sane mind will
> rename them
> > and make them conflict (because they are not parallel-installable).
> >
> > I wanted to also add ruby bindings to a module, but I never got time
> > to actually do it.
> >
> > # What about dependencies change?
> >
> > Let's not lock ourselves into libgit2 story, just take an abstract names.
> >
> > Module foo:rolling depends on bar:1. Name of stream "rolling" means to
> serve
> > purpose of "user-focused content meant for general use". That means,
> > if foo's upstream decided to update their bar dependency to a new
> version,
> > foo:rolling maintainer would just switch dependency to bar:2.
>
> AIUI, the issue here is that *there should not be* bar:1 and bar:2.
> Module streams are supposed to be 'functional' (as in your 'rolling'
> example). They are not supposed to be version numbers. This needs to be
> more clearly explained in the guidelines and enforced, but the way the
> modularity concept turned out, version-based module streams are *wrong*
> and should not exist. I recall earlier designs along the way actually
> sort of envisaged version-based module streams, but that is not how
> it's supposed to work in the final design.
>

Versioned module streams should definitely exist, that's one of the main
points of Modularity. But there are certainly exceptions.

To keep the expectations of Fedora's stable ABI within a release, we can't
change the default stream of a module mind-release. I know, that's probably
clear and that's not the issue here. But building on that, at the same
time, we can't let "dnf update" to change streams on your system
mid-release, because that would be basically breaking the ABI expectations
as well — different mechanics, same problem.

So in this specific example — where upstream is changing the ABI very often
— freezing that package to one major version per Fedora release doesn't
work. Especially when different packages need a different version at the
same time. So streams are probably not the right way to deal with this
specific case. Streams are a great way to provide our users a choice. A
choice of one of multiple versions of software (mostly leaf applications)
that are natively built and maintained for their fedora release (as opposed
to enabling rawhide repos and I don't know what to get a different
version). But it's not a solution for providing multiple versions of
libraries that need to be installed at the same time.

When different packages require different version of a library at the same
time, we can definitely use existing mechanisms for parallel installation
such as providing different packages with different names, installing the
library into different places, like compat-packages.

What Modularity could help you with — in an arbitrary case when there is
bar:1 and bar:2 — is to build your "rolling" module (and other modules)
against both of those using stream expansion, providing two different
binaries of the same module stream (with different context), and letting
DNF to install the right one based on what is on your system, or based on
defaults. This is one of the new capabilities Modularity brings to the
table. Of course, that only works if your "rolling" module can be actually
built against both.

But for cases when your "rolling" module can't be built against both
versions, existing mechanisms for parallel installation should be used
instead. Modularity doesn't reimplement those existing mechanisms.





> --
> Adam Williamson
> Fedora QA Community Monkey
> IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
> http://www.happyassassin.net
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>


-- 

Adam Šamalík
---
Senior Software Engineer
Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Modularity vs. libgit

2019-06-21 Thread Pete Walter


20.06.2019, 22:50, "Igor Gnatenko" :
> Hello,
>
> I just wanted to give you an update from my last discussions on
> #fedora-modularity and other places.
>
> # Problems definition
>
> * Default modules can't have conflicting dependenices
> * Changing dependencies in a stream is not supported
>
> # Why does libgit2 has to be a module?
>
> libgit2 is not just one package. It is an ecosystem.
>
> Right now libgit2 module provides libgit2 itself and python bindings.
> While we can obviously provide libgit2_0.26, libgit2_0.27 and such,
> this does not help us with python packages. Nobody in sane mind will rename 
> them
> and make them conflict (because they are not parallel-installable).
>
> I wanted to also add ruby bindings to a module, but I never got time
> to actually do it.

I'm the python-pygit2 maintainer (libgit2 python bindings) and I'm very much 
opposed to what you are doing with libgit2 and modules. It's a huge mess right 
now. Please go back to a known libgit2 version shipped in one Fedora, so I know 
which version to target with the python bindings (and so that apps have a clear 
story as well what to expect).

Also, I'd like you to stop shipping the python bindings inside the libgit2 
module. They are working very well as a regular package. Thanks.

The parallel installable libgit2 non-modular packages that were proposed 
earlier sounds like a nice solution. I'd be happy to help implement that.

Pete
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-20 Thread Adam Williamson
On Fri, 2019-06-21 at 01:10 +0200, Fabio Valentini wrote:
> 
> But ... if it's not for different version streams, then what *is it* for? 樂
> (What about the plans to offer different versions of e.g. NodeJS via
> streams? Is that wrong then, too?)
> 
> Being able to offer different versions is the only useful use-case I can
> think of, and if that's not the intended usage ... I don't know why we need
> it, or why somebody should use it at all ...

Eh, I should probably butt out before I get things too far wrong. Let's
wait for a licensed Modularity Person to show up and explain :)
Stephen? Someone?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-20 Thread Fabio Valentini
On Fri, Jun 21, 2019, 00:55 Adam Williamson 
wrote:

> On Thu, 2019-06-20 at 23:48 +0200, Igor Gnatenko wrote:
> > Hello,
> >
> > I just wanted to give you an update from my last discussions on
> > #fedora-modularity and other places.
> >
> > # Problems definition
> >
> > * Default modules can't have conflicting dependenices
> > * Changing dependencies in a stream is not supported
> >
> > # Why does libgit2 has to be a module?
> >
> > libgit2 is not just one package. It is an ecosystem.
> >
> > Right now libgit2 module provides libgit2 itself and python bindings.
> > While we can obviously provide libgit2_0.26, libgit2_0.27 and such,
> > this does not help us with python packages. Nobody in sane mind will
> rename them
> > and make them conflict (because they are not parallel-installable).
> >
> > I wanted to also add ruby bindings to a module, but I never got time
> > to actually do it.
> >
> > # What about dependencies change?
> >
> > Let's not lock ourselves into libgit2 story, just take an abstract names.
> >
> > Module foo:rolling depends on bar:1. Name of stream "rolling" means to
> serve
> > purpose of "user-focused content meant for general use". That means,
> > if foo's upstream decided to update their bar dependency to a new
> version,
> > foo:rolling maintainer would just switch dependency to bar:2.
>
> AIUI, the issue here is that *there should not be* bar:1 and bar:2.
> Module streams are supposed to be 'functional' (as in your 'rolling'
> example). They are not supposed to be version numbers. This needs to be
> more clearly explained in the guidelines and enforced, but the way the
> modularity concept turned out, version-based module streams are *wrong*
> and should not exist. I recall earlier designs along the way actually
> sort of envisaged version-based module streams, but that is not how
> it's supposed to work in the final design.
>

But ... if it's not for different version streams, then what *is it* for? 樂
(What about the plans to offer different versions of e.g. NodeJS via
streams? Is that wrong then, too?)

Being able to offer different versions is the only useful use-case I can
think of, and if that's not the intended usage ... I don't know why we need
it, or why somebody should use it at all ...

Fabio

-- 
> Adam Williamson
> Fedora QA Community Monkey
> IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
> http://www.happyassassin.net
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-20 Thread Adam Williamson
On Thu, 2019-06-20 at 23:48 +0200, Igor Gnatenko wrote:
> Hello,
> 
> I just wanted to give you an update from my last discussions on
> #fedora-modularity and other places.
> 
> # Problems definition
> 
> * Default modules can't have conflicting dependenices
> * Changing dependencies in a stream is not supported
> 
> # Why does libgit2 has to be a module?
> 
> libgit2 is not just one package. It is an ecosystem.
> 
> Right now libgit2 module provides libgit2 itself and python bindings.
> While we can obviously provide libgit2_0.26, libgit2_0.27 and such,
> this does not help us with python packages. Nobody in sane mind will rename 
> them
> and make them conflict (because they are not parallel-installable).
> 
> I wanted to also add ruby bindings to a module, but I never got time
> to actually do it.
> 
> # What about dependencies change?
> 
> Let's not lock ourselves into libgit2 story, just take an abstract names.
> 
> Module foo:rolling depends on bar:1. Name of stream "rolling" means to serve
> purpose of "user-focused content meant for general use". That means,
> if foo's upstream decided to update their bar dependency to a new version,
> foo:rolling maintainer would just switch dependency to bar:2.

AIUI, the issue here is that *there should not be* bar:1 and bar:2.
Module streams are supposed to be 'functional' (as in your 'rolling'
example). They are not supposed to be version numbers. This needs to be
more clearly explained in the guidelines and enforced, but the way the
modularity concept turned out, version-based module streams are *wrong*
and should not exist. I recall earlier designs along the way actually
sort of envisaged version-based module streams, but that is not how
it's supposed to work in the final design.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-20 Thread Igor Gnatenko
Hello,

I just wanted to give you an update from my last discussions on
#fedora-modularity and other places.

# Problems definition

* Default modules can't have conflicting dependenices
* Changing dependencies in a stream is not supported

# Why does libgit2 has to be a module?

libgit2 is not just one package. It is an ecosystem.

Right now libgit2 module provides libgit2 itself and python bindings.
While we can obviously provide libgit2_0.26, libgit2_0.27 and such,
this does not help us with python packages. Nobody in sane mind will rename them
and make them conflict (because they are not parallel-installable).

I wanted to also add ruby bindings to a module, but I never got time
to actually do it.

# What about dependencies change?

Let's not lock ourselves into libgit2 story, just take an abstract names.

Module foo:rolling depends on bar:1. Name of stream "rolling" means to serve
purpose of "user-focused content meant for general use". That means,
if foo's upstream decided to update their bar dependency to a new version,
foo:rolling maintainer would just switch dependency to bar:2.

However, with modularity once you enable stream
(and "bar:1" gets enabled implicitly), you stay on it forever. This means
that maintainer of "foo" just can't change dependencies ever. This is just bad.

I think solution here is to teach DNF about implicitly enabled modules
and switch streams for them when needed. Or even add flag "--i-know-what-i-do"
for the time being. From what I understand, this has been done so that
systems do not break ever… However, with right RPM dependencies, I think this
should not be a problem. And after all, RHEL can be different from Fedora.
Right now DNF resolves dependencies of modules separately from RPM ones
(which is yet another thing worries me[0]). If we teach it to do it at once,
we can be sure that whatever is installed locally by RPM, will not have
broken dependencies so it will prevent switching stream.

# Why should defaults conflict?

Let's think what defaults mean and why they even exist. People used to do
"dnf install foo" and get their package. With modularity, they would need to
explicitly enable some module. Here we come to the defaults. Packages from
default streams are available for installation without explicit enabling of
modules. Installing the actual package will trigger enablement of a
module stream
(so that it does not get upgraded to an incompatible version).

In Fedora we try to make as much as possible to be parallel-installable.
However, this is not always possible. Conflicts happen everywhere. If
foo and bar
can't be installed in parallel, we do not just abort when not installing
both of them at the same time. You can install foo separately from bar.
This is unfortunate, but that's what it is.

Why modules have to be different? Default modules are there to make
packages available for selection. When user will try to install them
and there is a conflict, only then DNF should show an error. Not in any other
cases.

# What should we do now with modularity and libgit?

I can change libgit2 module to provide static subpackages and teach
Rust applications to link statically so we won't have these dependencies.

But this is temporary workaround because we don't have any automation for
rebuilding modules or rpms on a dependency change.

What I think should happen is DNF team to provide an ETA to fix issues above
and depending on that we can see whether we should take this path or just
throw away modularity and focus on other solutions which are not that invasive.




[0] https://bugzilla.redhat.com/show_bug.cgi?id=1677746

On Thu, Jun 13, 2019 at 9:22 PM Adam Samalik  wrote:
>
> So, I'd like to discuss the libgit issue [1] [2] we're experiencing. With a 
> help of a few people, I've put together this post to get us on common ground: 
> https://communityblog.fedoraproject.org/modularity-vs-libgit/
>
> There are few ideas about solving the issue right now. But we might be able 
> to think about better ways to deal with similar issues long-term. Let's do 
> this!
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1717117
> [2] https://pagure.io/fesco/issue/2146#comment-575852
> --
>
> Adam Šamalík
> ---
> Senior Software Engineer
> Red Hat
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fe

Re: Modularity vs. libgit

2019-06-18 Thread Nicolas Mailhot via devel

Le 2019-06-17 22:04, Terry Bowling a écrit :

Hi

If I may, since I also represented the customer side not so long ago in 
a fortunexxx company.



* Customers think we had too many repos.  It is hard to find what
they need.


From a customer point of view you sidestepped the demand. Instead of 
making the necessary integration work and arbitrations to merge back all 
those awful optional repos in the main RHEL repo, you stashed them in 
modules.


Don't expect a lot of customer happiness about it.


* Customers need more options that are fully supported to meet their
production workload requirements.


ie you don't want to make integration decisions, so you say customers 
want different things. Or course they want different things. They're 
human. But mostly they want a single, unified, reliable RHEL baseline, 
not a jungle of optional conflicting repos, not a jungle of optional 
conflicting modules.


If a single RHEL baseline provides too little choice, maybe you need to 
release new RHEL major versions more often to provide a wider spectrum 
of options. Certainly, jungling a set of bi-yearly RHEL major version 
stream, would be less awful customer side than jungling scores of 
conflicting optional repos or modules.


It will end up with more and more customers giving up on those optional 
things, and using EPEL directly, reducing the scope of thing they pay 
support for RH-side, and reducing the amount of money they feel they owe 
to RH.


Regards,

--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-18 Thread Dridi Boukelmoune
On Mon, Jun 17, 2019 at 8:59 PM Terry Bowling  wrote:
>
> Oh, I forgot to add that related to parallel installations, when conflicting 
> modules are desired, generally containerization or virtualization is the 
> recommended solution.  However, this is from the RHEL user persona 
> perspective.  We realize that is a very different user persona from say a 
> developer working in Fedora Rawhide.

Every once in a while someone will accuse Red Hat of using Fedora as a
test bed for RHEL, and someone else will insist that despite Red Hat
being a (the?) major sponsor of the project it is still a community
project. But that kind of explanation can only reinforce the test bed
idea.

I don't want to prevent Red Hat from serving their customers better,
otherwise I would be shooting myself in the foot considering that for
$DAYJOB I deal with customers running RHEL or derivatives. And one
reason why I chose Fedora (over say, Ubuntu) for my personal use is
because I felt more comfortable on RHEL systems than on Debian
derivatives on previous jobs, and looking at other distribution models
(Gentoo, Slackware, Arch...) I didn't find enough appeal.

But if Fedora is not a test bed, and yet RHEL is downstream of Fedora,
RHEL could have implemented its modularity without forcing it onto
Fedora. And the worst part for me is that it was obvious from the
start that we would run into that kind of problems because Fedora has
a strong anti-bundling policy (which I personally approve) and "leaf"
packages eventually indirectly conflict because of dependencies.

I don't know RHEL policies, but Red Hat could certainly allow some
form of bundling to avoid module conflicts and not drag Fedora into a
modularity "mess" that is incompatible with its Four principles. But I
guess that ship has sailed and now it's too late...


Dridi
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-17 Thread Colin Walters


On Mon, Jun 17, 2019, at 2:51 PM, Neal Gompa wrote:

> RPM-OSTree is functionally irrelevant in this discussion, 

Changing how we handle the kernel is certainly relevant.

> since it has
> its own behavior patterns and eschews compatibility with the greater
> ecosystem anyway. 

I don't think that's a fair characterization, we obviously go to a fair amount 
of effort to support some of the existing ecosystem, but achieving the goals 
conflicts with 100% compatibility.

> It doesn't even support modules,

We will at some point:
https://github.com/projectatomic/rpm-ostree/issues/1435

In practice today for people doing custom builds on the server side we usually 
want more sophistication than the default module UX anyways (e.g. we just added 
lockfiles: https://github.com/projectatomic/rpm-ostree/pull/1745 ), or we just 
want the latest (what the current git master does).

>  so this whole discussion doesn't even matter for systems built on RPM-OSTree.

Not at all!  We expect and encourage people to use dnf and modules inside 
containers.  At some point later we will probably add some support on the host 
for layering but I think it's more important to flesh out the toolbox approach 
for living inside a container.


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-17 Thread Terry Bowling
Oh, I forgot to add that related to parallel installations, when
conflicting modules are desired, generally containerization or
virtualization is the recommended solution.  However, this is from the RHEL
user persona perspective.  We realize that is a very different user persona
from say a developer working in Fedora Rawhide.

Thank you,
Terry Bowling
Sr. Technical Product Manager - RHEL Systems Mgmt
Red Hat, Inc.



On Mon, Jun 17, 2019 at 4:04 PM Terry Bowling  wrote:

> Regarding to a few of the questions about why modularity was created in
> the first place (paraphrased), I can offer the following background.  Note,
> I was not on the team at the very beginning but have been very involved for
> the last ~2 years.
>
> Do not consider the following to be formal answers, guidance, or laws of
> nature.  I'm only trying to help share some perspective.
>
> From the RHEL Product Management (PM) side, we observed the following
> customer+market problems.  Note this is not a perfect list but a rough,
> paraphrased, quick brain dump:
>
>- Customers think we had too many repos.  It is hard to find what they
>need.
>   - optional, supplementary, extras, software collections, and more...
>- Customers need more options that are fully supported to meet their
>production workload requirements.
>   - 1. "I need as little change as possible.  I will stick with the
>   version of DB provided with X.0 release for the next 10 years."
>   - 2. "I need newer versions of xyz to benefit from some features."
>   - 3. A few states in between 1 & 2.
>- "Software Collections are helpful and provide us more choices, but
>the user experience is awkward and most of the time we only install 1
>version at a time, with the exception of a few programming language
>environments (multiple python, gcc, and so forth)."  And it's a different
>repo and a non-native software management workflow.
>- RH need the distinction to provide realistic life cycles that
>matched the true upstream life cycle.  5 year life cycles of DB's are a
>good example, as well as the python 2 reaching its end of life.
>Application Stream modules help with defining this, as well as enabling
>customers to block access to versions that have reached the end of their
>life cycle.
>
> Thus, RHEL PM asked for a method so solve these problems above.  PM
> objectives ideally should not dictate the technical implementation but
> rather focus on addressing the customer/market problems, user experience,
> supportability, and things like that.  My understanding was that
> package-name-versioning could have been acceptable to PM, but there were a
> number of support and engineering challenges which led to the modularity
> design.
>
> That said, some of the internal guidance was that:
>
>- Only select modules are parallel installable, primarily programming
>languages such as python.
>- while modularity could help with some traditional repository
>challenges, it does not solve all, nor does it make RPM dependency problems
>go away.  Therefore only select things in Appstream should be modularized,
>usually when collections of rpms serve a common use case, such as databases
>or Identity Management.
>   - The RHEL distribution ensures a sane state of compatibility to
>   ensure a good user experience.  So the analogy of conflicts with many
>   different repos does not magically go away.
>   - Disciplined distribution content management policies and
>   compatibility are still required as if they were different repos.
>- Default stream versions defined so that if a user wants a simple
>RHEL 7 like experience, they have it.  They are not forced to use
>modularity unless they want to or their workload dictates different 
> modules.
>- Changing streams is a manual process, at least until modularity
>matures.  If a customer chooses a stream for their workload, we cannot
>break that customer's workload.  Though shalt do no harm.  Therefore
>auto-stream switching is off-limits in RHEL.  At least for now until we
>further understand customer demand and the risk impact as Appstream grows
>to include more modules.
>
> That's all I can think of right now and I might have missed some things.
> Again, it does NOT answer many of the questions in this thread and we know
> we have future work to do.  I would be happy to consult with, but not
> dictate to, Fedora members on defining their own policy guidelines if
> desired.
>
> Thank you,
> Terry Bowling
> Sr. Technical Product Manager - RHEL Systems Mgmt
> Red Hat, Inc.
>
>
>
> On Mon, Jun 17, 2019 at 2:52 PM Neal Gompa  wrote:
>
>> On Mon, Jun 17, 2019 at 2:29 PM Colin Walters  wrote:
>> >
>> > On Mon, Jun 17, 2019, at 4:47 AM, Michael Schroeder wrote:
>> > > On Fri, Jun 14, 2019 at 12:12:01PM -0400, Neal Gompa wrote:
>> > > > I would actually really like to see rpm's multiversioning
>> 

Re: Modularity vs. libgit

2019-06-17 Thread Terry Bowling
Regarding to a few of the questions about why modularity was created in the
first place (paraphrased), I can offer the following background.  Note, I
was not on the team at the very beginning but have been very involved for
the last ~2 years.

Do not consider the following to be formal answers, guidance, or laws of
nature.  I'm only trying to help share some perspective.

>From the RHEL Product Management (PM) side, we observed the following
customer+market problems.  Note this is not a perfect list but a rough,
paraphrased, quick brain dump:

   - Customers think we had too many repos.  It is hard to find what they
   need.
  - optional, supplementary, extras, software collections, and more...
   - Customers need more options that are fully supported to meet their
   production workload requirements.
  - 1. "I need as little change as possible.  I will stick with the
  version of DB provided with X.0 release for the next 10 years."
  - 2. "I need newer versions of xyz to benefit from some features."
  - 3. A few states in between 1 & 2.
   - "Software Collections are helpful and provide us more choices, but the
   user experience is awkward and most of the time we only install 1 version
   at a time, with the exception of a few programming language environments
   (multiple python, gcc, and so forth)."  And it's a different repo and a
   non-native software management workflow.
   - RH need the distinction to provide realistic life cycles that matched
   the true upstream life cycle.  5 year life cycles of DB's are a good
   example, as well as the python 2 reaching its end of life.  Application
   Stream modules help with defining this, as well as enabling customers to
   block access to versions that have reached the end of their life cycle.

Thus, RHEL PM asked for a method so solve these problems above.  PM
objectives ideally should not dictate the technical implementation but
rather focus on addressing the customer/market problems, user experience,
supportability, and things like that.  My understanding was that
package-name-versioning could have been acceptable to PM, but there were a
number of support and engineering challenges which led to the modularity
design.

That said, some of the internal guidance was that:

   - Only select modules are parallel installable, primarily programming
   languages such as python.
   - while modularity could help with some traditional repository
   challenges, it does not solve all, nor does it make RPM dependency problems
   go away.  Therefore only select things in Appstream should be modularized,
   usually when collections of rpms serve a common use case, such as databases
   or Identity Management.
  - The RHEL distribution ensures a sane state of compatibility to
  ensure a good user experience.  So the analogy of conflicts with many
  different repos does not magically go away.
  - Disciplined distribution content management policies and
  compatibility are still required as if they were different repos.
   - Default stream versions defined so that if a user wants a simple RHEL
   7 like experience, they have it.  They are not forced to use modularity
   unless they want to or their workload dictates different modules.
   - Changing streams is a manual process, at least until modularity
   matures.  If a customer chooses a stream for their workload, we cannot
   break that customer's workload.  Though shalt do no harm.  Therefore
   auto-stream switching is off-limits in RHEL.  At least for now until we
   further understand customer demand and the risk impact as Appstream grows
   to include more modules.

That's all I can think of right now and I might have missed some things.
Again, it does NOT answer many of the questions in this thread and we know
we have future work to do.  I would be happy to consult with, but not
dictate to, Fedora members on defining their own policy guidelines if
desired.

Thank you,
Terry Bowling
Sr. Technical Product Manager - RHEL Systems Mgmt
Red Hat, Inc.



On Mon, Jun 17, 2019 at 2:52 PM Neal Gompa  wrote:

> On Mon, Jun 17, 2019 at 2:29 PM Colin Walters  wrote:
> >
> > On Mon, Jun 17, 2019, at 4:47 AM, Michael Schroeder wrote:
> > > On Fri, Jun 14, 2019 at 12:12:01PM -0400, Neal Gompa wrote:
> > > > I would actually really like to see rpm's multiversioning
> capabilities
> > > > extended to support this.
> > >
> > > I'd actually prefer to drop the multiversion mode for the kernel and
> > > instead add the version to the kernel package name.
> >
> > FWIW in rpm-ostree we go to some lengths to explicitly undo the libdnf
> default of multiple versions for the kernel, because the ostree side of
> things effectively multi-versions all of userspace as well.   We're always
> creating a new root, so there's no concern about removing modules from the
> running kernel (any more than there is a similar concern for userspace
> components).  An ostree deployment includes a 

Re: Modularity vs. libgit

2019-06-17 Thread Neal Gompa
On Mon, Jun 17, 2019 at 2:29 PM Colin Walters  wrote:
>
> On Mon, Jun 17, 2019, at 4:47 AM, Michael Schroeder wrote:
> > On Fri, Jun 14, 2019 at 12:12:01PM -0400, Neal Gompa wrote:
> > > I would actually really like to see rpm's multiversioning capabilities
> > > extended to support this.
> >
> > I'd actually prefer to drop the multiversion mode for the kernel and
> > instead add the version to the kernel package name.
>
> FWIW in rpm-ostree we go to some lengths to explicitly undo the libdnf 
> default of multiple versions for the kernel, because the ostree side of 
> things effectively multi-versions all of userspace as well.   We're always 
> creating a new root, so there's no concern about removing modules from the 
> running kernel (any more than there is a similar concern for userspace 
> components).  An ostree deployment includes a (kernel+initramfs,userspace) as 
> a pair; two deployments can happen to share a kernel, or not.   A bit more 
> info in
> https://github.com/projectatomic/rpm-ostree/pull/1346

RPM-OSTree is functionally irrelevant in this discussion, since it has
its own behavior patterns and eschews compatibility with the greater
ecosystem anyway. It doesn't even support modules, so this whole
discussion doesn't even matter for systems built on RPM-OSTree.


-- 
真実はいつも一つ!/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-17 Thread Samuel Sieb

On 6/14/19 6:29 AM, Daniel Mach wrote:

Dne 14. 06. 19 v 6:23 Samuel Sieb napsal(a):
After reading the bug report and the discussions, I still don't 
understand why dnf is complaining about a conflict with packages 
(modules?) that are not installed and are not even trying to be 
installed.  Can someone explain that?


The dependency resolution is done on 2 levels.
RPM dependency resolution still works the same even with modules.

What's different on a modular system is a pre-processing step that 
resolves modules and filters out RPMs that do not match with 
enabled/default streams.


What makes it even more complicated is stream expansion:
https://docs.fedoraproject.org/en-US/modularity/architecture/stream-expansion/ 


Some modules get build several times against multiple dependency sets.
These dependency sets are captured as Context field in module metadata.
DNF has to resolve the contexts and pick the right ones based on enabled 
streams. This is done for *every* module version (not just the latest) 
in order to compute which RPMs will be available to the user and which 
not. And this causes the conflicts you were asking about.


But I still don't understand why it's looking at modules that are not 
installed and not asked to be installed.  If they aren't getting 
installed, how is there a conflict?  This seems to eliminate a lot of 
the benefit of modules.

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-17 Thread Colin Walters


On Mon, Jun 17, 2019, at 4:47 AM, Michael Schroeder wrote:
> On Fri, Jun 14, 2019 at 12:12:01PM -0400, Neal Gompa wrote:
> > I would actually really like to see rpm's multiversioning capabilities
> > extended to support this.
> 
> I'd actually prefer to drop the multiversion mode for the kernel and
> instead add the version to the kernel package name.

FWIW in rpm-ostree we go to some lengths to explicitly undo the libdnf default 
of multiple versions for the kernel, because the ostree side of things 
effectively multi-versions all of userspace as well.   We're always creating a 
new root, so there's no concern about removing modules from the running kernel 
(any more than there is a similar concern for userspace components).  An ostree 
deployment includes a (kernel+initramfs,userspace) as a pair; two deployments 
can happen to share a kernel, or not.   A bit more info in
https://github.com/projectatomic/rpm-ostree/pull/1346
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-17 Thread Josh Boyer
On Sun, Jun 16, 2019 at 2:53 AM Remi Collet  wrote:
>
> Le 14/06/2019 à 20:03, Josh Boyer a écrit :
> > On Fri, Jun 14, 2019 at 3:50 AM Remi Collet  
> > wrote:
> >>
> >>
> >>> IMHO, having library in modules is an error, this can only raise issues
> >>
> >> Some examples, taken from RHEL-8
> >>
> >> libJudy is part of the mariadb module ... (without -devel)
> >> libssh2 is part of the virt module... (without -devel)
> >> libzip is part of the php module...
> >> etc
> >>
> >> How are we going to manage this in EPEL ?
> >
> > Can you elaborate further?
>
> How are you going to build package requiring libssh2 in EPEL ?
>
> Dependency on "virt" seems terribly strange
> and we don't have the "devel" package, and are not allowed per
> Guidelines to replace things which are part of RHEL.
>
> So probably we have to add libssh2 in EPEL, but what will happen if
> soname change ? making installation if "virt" module impossible ?
>
> As "libssh2" is only used by "virt", and is not part of the provided
> contract of the module, we can imagine
>
> - virt version x with libssh2 version 1
> - virt version y (major update = new stream) with libssh2 version 2
> - virt version y (minor update, same stream) with libssh2 version 3
> - etc
>
>
> Next problem, I need libssh2 and libjudy (and some others) in the PHP
> module, so ?
>
> - add a module dependency on mariadb and virt ?
>
> => this can't work and don't make sense
>
> - add the judy and libssh2 and other lib in the php module ?
>
> => at some point, we are going to generate conflicts between modules

Why can't libssh2 be packaged into a module of its own in EPEL?  Yes,
this looks odd at first, but it doesn't expressly replace the RHEL
version, and it allows you to have module dependencies that do make
sense.

Granted, that may mean modules could conflict but in this specific
case I'm not immediately seeing that.  If the soname changes in the
libssh2 module, the installation files would also change and the virt
module could continue using its own copy, right?

> IMHO, such libraries have to be in the base distribution.

Ah, well in many cases, yes.  There are cases where a library is
considered an internal implementation detail and not for general usage
outside of the core functionality it was included for.  Modules
actually make this possible in a way, with the pitfalls you're
highlighting.  The usecases also differ greatly between one OS and
another, so what makes sense in e.g. RHEL might not make sense in
Fedora or even EPEL.

josh

> Another funny thing, I have some PHP extensions which use nodejs, how to
> mange this in a modular world ?
>
> - make PHP module depends on NODEJS module ?
>
> - create another module only for these extensions ?
>
>
>
> For memory, PHP is not available as module in Fedora, because of such
> issues (and PHP stack is hundreds of packages, with tons of library
> bindings).
>
> But I really want to see more PHP stuff in EPEL, as for previous
> version, and this have to be managed as module.
>
>
>
>
> Remi
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-17 Thread Nico Kadel-Garcia
On Mon, Jun 17, 2019 at 5:35 AM Michael Schroeder  wrote:
>
> On Fri, Jun 14, 2019 at 12:12:01PM -0400, Neal Gompa wrote:
> > I would actually really like to see rpm's multiversioning capabilities
> > extended to support this.
>
> I'd actually prefer to drop the multiversion mode for the kernel and
> instead add the version to the kernel package name.
>
> Cheers,
>   Michael.

This would screw up a lot of existing tools and procedures, and screw
up ordinary updating. I wouldn't recommend it.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-17 Thread Neal Gompa
On Mon, Jun 17, 2019 at 5:35 AM Michael Schroeder  wrote:
>
> On Fri, Jun 14, 2019 at 12:12:01PM -0400, Neal Gompa wrote:
> > I would actually really like to see rpm's multiversioning capabilities
> > extended to support this.
>
> I'd actually prefer to drop the multiversion mode for the kernel and
> instead add the version to the kernel package name.
>

It leads to a poorer experience managing those packages when they
aren't multiversioned. That's why OpenMandriva switched to
multiversioned packages for Lx4.

The experience managing kernel packages in Debian, Mageia, and other
distributions that do it the way you propose make things very
difficult and annoying.

openSUSE's multiversion packages are slightly more complex since they
are also multi-flavor (without multiple flavors) and are technically
invalid EVR-wise...

It might be nicer if they dropped the flavor bits...



--
真実はいつも一つ!/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-17 Thread Michael Schroeder
On Fri, Jun 14, 2019 at 12:12:01PM -0400, Neal Gompa wrote:
> I would actually really like to see rpm's multiversioning capabilities
> extended to support this.

I'd actually prefer to drop the multiversion mode for the kernel and
instead add the version to the kernel package name.

Cheers,
  Michael.

-- 
Michael Schroeder   m...@suse.de
SUSE LINUX GmbH,   GF Jeff Hawn, HRB 16746 AG Nuernberg
main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);}
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-17 Thread Miro Hrončok

On 14. 06. 19 6:27, Carl George wrote:

I think the best option is to create non-modular compat packages.  In my 
opinion, modularity makes sense for end user applications, but I'm not sure 
what benefits it has for libraries.  Libraries tend to work well as compat 
packages, so I implemented this in copr to try it out.

* https://copr.fedorainfracloud.org/coprs/carlwgeorge/parallel-libgit2/
* 
https://copr-be.cloud.fedoraproject.org/results/carlwgeorge/parallel-libgit2/fedora-rawhide-x86_64/00935643-libgit2_0.26/libgit2_0.26.spec
* 
https://copr-be.cloud.fedoraproject.org/results/carlwgeorge/parallel-libgit2/fedora-rawhide-x86_64/00935642-libgit2_0.27/libgit2_0.27.spec

This copr allows the following non-modular packages to be installed at the same 
time:

* Rawhide
 * libgit2   0.28.2
 * libgit2_0.27  0.27.8
 * libgit2_0.26  0.26.8

* Fedora 30
 * libgit2   0.27.8
 * libgit2_0.26  0.26.8

These packages follow the current naming and conflict packaging guidelines.

* https://docs.fedoraproject.org/en-US/packaging-guidelines/Naming/#multiple
* 
https://docs.fedoraproject.org/en-US/packaging-guidelines/Conflicts/#_compat_package_conflicts

If I'm missing anything about the benefits of a modular libgit2 help me 
understand.  Given the current issues, this seems like a reasonable solution.  
If other agree, I'm happy to submit these compat packages for review.


Thanks for providing a solution.

Obviously, the libgit maintainer(s) might argue that they want to maintain 
libgit in modules.




(The rest is not a direct reply, but rather my comment to the entire thread.)

I think we've come to a break point:

1. everything just works fine without modules
2. packagers are tempted by modularity
3. they move stuff to modules
4. everything is broken
5. best solution: go back to 1

(It's hard to not be sarcastic here and not congratulate ourselves on this huge 
success.)


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-16 Thread Nico Kadel-Garcia
On Sun, Jun 16, 2019 at 7:15 AM Kevin Kofler  wrote:
>
> Adam Samalik wrote:
> > So, I'd like to discuss the libgit issue [1] [2] we're experiencing. With
> > a help of a few people, I've put together this post to get us on common
> > ground: https://communityblog.fedoraproject.org/modularity-vs-libgit/
> >
> > There are few ideas about solving the issue right now. But we might be
> > able to think about better ways to deal with similar issues long-term.
> > Let's do this!
> >
> > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1717117
> > [2] https://pagure.io/fesco/issue/2146#comment-575852
>
> As I had posted in a blog comment:
>
> This is exactly the dependency hell (different packages/modules depending on
> incompatible versions of the same module) that I had been warning about ever
> since modularity has first been discussed in Fedora. I pointed out from day
> one that this is a showstopper that makes modularity entirely impractical
> and unsupportable. My concerns were dismissed or ignored. Now you are seeing
> exactly the situation that I had been warning about from day one.
>
> Kevin Kofler

As best I can tell, most of the dependency nightmares in Fedora and
RHEL 8 can be resolved, *so far*, by disabling the insistence on
"best" matching in mock. I ran into the modular component dependency
mismatches, hard, when backporting Samba 4.10.4 to RHEL 8. I'm not
sure how broken this is going to be for actually deploying RPMs.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-16 Thread Kevin Kofler
Adam Samalik wrote:
> So, I'd like to discuss the libgit issue [1] [2] we're experiencing. With
> a help of a few people, I've put together this post to get us on common
> ground: https://communityblog.fedoraproject.org/modularity-vs-libgit/
> 
> There are few ideas about solving the issue right now. But we might be
> able to think about better ways to deal with similar issues long-term.
> Let's do this!
> 
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1717117
> [2] https://pagure.io/fesco/issue/2146#comment-575852

As I had posted in a blog comment:

This is exactly the dependency hell (different packages/modules depending on 
incompatible versions of the same module) that I had been warning about ever 
since modularity has first been discussed in Fedora. I pointed out from day 
one that this is a showstopper that makes modularity entirely impractical 
and unsupportable. My concerns were dismissed or ignored. Now you are seeing 
exactly the situation that I had been warning about from day one.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-16 Thread Remi Collet
Le 14/06/2019 à 20:03, Josh Boyer a écrit :
> On Fri, Jun 14, 2019 at 3:50 AM Remi Collet  wrote:
>>
>>
>>> IMHO, having library in modules is an error, this can only raise issues
>>
>> Some examples, taken from RHEL-8
>>
>> libJudy is part of the mariadb module ... (without -devel)
>> libssh2 is part of the virt module... (without -devel)
>> libzip is part of the php module...
>> etc
>>
>> How are we going to manage this in EPEL ?
> 
> Can you elaborate further?

How are you going to build package requiring libssh2 in EPEL ?

Dependency on "virt" seems terribly strange
and we don't have the "devel" package, and are not allowed per
Guidelines to replace things which are part of RHEL.

So probably we have to add libssh2 in EPEL, but what will happen if
soname change ? making installation if "virt" module impossible ?

As "libssh2" is only used by "virt", and is not part of the provided
contract of the module, we can imagine

- virt version x with libssh2 version 1
- virt version y (major update = new stream) with libssh2 version 2
- virt version y (minor update, same stream) with libssh2 version 3
- etc


Next problem, I need libssh2 and libjudy (and some others) in the PHP
module, so ?

- add a module dependency on mariadb and virt ?

=> this can't work and don't make sense

- add the judy and libssh2 and other lib in the php module ?

=> at some point, we are going to generate conflicts between modules


IMHO, such libraries have to be in the base distribution.


Another funny thing, I have some PHP extensions which use nodejs, how to
mange this in a modular world ?

- make PHP module depends on NODEJS module ?

- create another module only for these extensions ?



For memory, PHP is not available as module in Fedora, because of such
issues (and PHP stack is hundreds of packages, with tons of library
bindings).

But I really want to see more PHP stuff in EPEL, as for previous
version, and this have to be managed as module.




Remi
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-14 Thread Josh Boyer
On Fri, Jun 14, 2019 at 3:50 AM Remi Collet  wrote:
>
>
> > IMHO, having library in modules is an error, this can only raise issues
>
> Some examples, taken from RHEL-8
>
> libJudy is part of the mariadb module ... (without -devel)
> libssh2 is part of the virt module... (without -devel)
> libzip is part of the php module...
> etc
>
> How are we going to manage this in EPEL ?

Can you elaborate further?

josh
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-14 Thread Nicolas Mailhot via devel

> We could go a step further and extend rpm and dnf to support multiple
> versions of same named packages for installation.  This is doable but
> not necessarily trivial. 

Having rescued this week a system in abysmal stale, with traces of rpm
forcing right and left, I'd say this would also require a robust way to
list all the installed versions, and operate on a specific set of them,
in rpm/dnf refresh/remove modes

And that would also require some thought on how to repoquery and
license audit. A version bump can mean rehosting and licensing changes,
not just another variant of the same code.

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-14 Thread Dridi Boukelmoune
> > Thanks to soname, library are perfect use case for parallel installation
> > of multiple versions.
>
> +1
>
> We could go a step further and extend rpm and dnf to support multiple
> versions of same named packages for installation.  This is doable but

Both rpm and dnf can do that, try running `rpm -qa kernel`. As long as
same-name packages don't conflict it's ok.

> not necessarily trivial.  Upgrades would need a way to specify what
> package NVR they are upgrading (doable) and dependencies, requires, and
> obsoletes would need to be reviewed to ensure you don't wipe out a
> version you want installed.  Plus more.  Solvable and the same end
> result for users, just a different approach.

The problem with parallel installation is that dnf (formerly yum) only
offers a narrow option for parallel installation. But probably a
plugin could do that.

The real problem is so-called modularity. I support the goal of
modularity, but not its vague name and neither its implementation from
what I could grok. As soon as two "leaf" modules share a dependency it
falls apart as this thread starting point shows.

Parallel-installable libraries, whether or not packages are named
following Debian's convention, are also not a problem. Either this is
explicitly supported by the build system (e.g. usually autotools and
pkg-config would) or otherwise it can be "forcefully enabled" in the
RPM spec. And that doesn't prevent us from having parallel-installable
devel packages too, but someone will likely have to give up the global
/usr/include namespace (or different stack equivalent).

The problem that fails every attempt at squaring dependencies is their
quadratic nature. In the case of lagging upstreams, the First
principle mandates that we help dependent upstream move forward. In
the legitimate case of multiple living "streams" (another word I find
too vague) where N streams may be supported simultaneously, then
parallel installation is a better solution, not just availability.

But but but... Not everyone can take the single global namespace at
once and nobody wants to run gccX.Y on Fedora or RHEL, and
update-alternatives (or insert any other attempt) doesn't bring the
convenience of running gcc and expect TheRightThing(tm) out of the
box. At best it delays the problems.



Dridi
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-14 Thread Carl George
> Or you think changing huge pieces of infrastructure, and working for
> 3+ years on a project (modularity) was done just to have 3 versions of
> Node.js available?

Of course not.  Modularity is a great fit to provide multiple versions many 
applications (in one repository), such as php, mariadb, postgresql, nginx, 
ruby, and more.  Suggesting that it is just about nodejs is a strawman argument.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-14 Thread Igor Gnatenko
On Fri, Jun 14, 2019 at 6:11 PM David Cantrell  wrote:
>
> On 6/14/19 2:52 AM, Remi Collet wrote:
> > Le 13/06/2019 à 20:31, Adam Samalik a écrit :
> >> So, I'd like to discuss the libgit issue [1] [2] we're experiencing. With a
> >> help of a few people, I've put together this post to get us on common
> >> ground: https://communityblog.fedoraproject.org/modularity-vs-libgit/
> >>
> >> There are few ideas about solving the issue right now. But we might be able
> >> to think about better ways to deal with similar issues long-term. Let's do
> >> this!
> >
> > IMHO, having library in modules is an error, this can only raise issues
> >
> > Perhaps debian was right, and we should use a naming schema matching the
> > library ABI, so including the soname
> >
> >   libgit26-0.26.8
> >   libgit27-0.27.8
> >   libgit28-0.28.1
> >   etc
> >
> > Thanks to soname, library are perfect use case for parallel installation
> > of multiple versions.
>
> +1
>
> We could go a step further and extend rpm and dnf to support multiple
> versions of same named packages for installation.  This is doable but
> not necessarily trivial.  Upgrades would need a way to specify what
> package NVR they are upgrading (doable) and dependencies, requires, and
> obsoletes would need to be reviewed to ensure you don't wipe out a
> version you want installed.  Plus more.  Solvable and the same end
> result for users, just a different approach.

I don't disagree, but why did we invent modularity then? It is
supposed to solve problems of parallel availability of software. And
if it does not and we try to do something else now, should we drop
modularity entirely?

>
> >
> > And thanks to range version, this can be describe for dependencies.
> >
> > BuildRequires: (pkgconfig(libgit) >=26 with pkgconfig(libgit) <= 27)
> >
> > Yes, this will means more packages, and more reviews, but having a new
> > review on a old library when soname change make sense.
> >
> > Of course, old versions have to be retired when no more used
> >
> > Modules are fine for applications.
> >
> >
> > Remi
> >
> >
> > P.S. yes compat_* name schema also works... but also consider it as a
> > bit ugly...
> > ___
> > devel mailing list -- devel@lists.fedoraproject.org
> > To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives: 
> > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> >
>
>
> --
> David Cantrell 
> Red Hat, Inc. | Boston, MA | EST5EDT
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-14 Thread Igor Gnatenko
I did not intend to provide solution to the problem, just as an idea.
Or you think changing huge pieces of infrastructure, and working for
3+ years on a project (modularity) was done just to have 3 versions of
Node.js available?

On Fri, Jun 14, 2019 at 4:36 PM Carl George  wrote:
>
> > So what is the point of modules then? If we want just multiple
> > versions of a few applications, we can just have few repositories.
>
> Add another repository for every alternate version (stream) doesn't scale.
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-14 Thread Neal Gompa
On Fri, Jun 14, 2019 at 12:05 PM David Cantrell  wrote:
>
> On 6/14/19 2:52 AM, Remi Collet wrote:
> > Le 13/06/2019 à 20:31, Adam Samalik a écrit :
> >> So, I'd like to discuss the libgit issue [1] [2] we're experiencing. With a
> >> help of a few people, I've put together this post to get us on common
> >> ground: https://communityblog.fedoraproject.org/modularity-vs-libgit/
> >>
> >> There are few ideas about solving the issue right now. But we might be able
> >> to think about better ways to deal with similar issues long-term. Let's do
> >> this!
> >
> > IMHO, having library in modules is an error, this can only raise issues
> >
> > Perhaps debian was right, and we should use a naming schema matching the
> > library ABI, so including the soname
> >
> >   libgit26-0.26.8
> >   libgit27-0.27.8
> >   libgit28-0.28.1
> >   etc
> >
> > Thanks to soname, library are perfect use case for parallel installation
> > of multiple versions.
>
> +1
>
> We could go a step further and extend rpm and dnf to support multiple
> versions of same named packages for installation.  This is doable but
> not necessarily trivial.  Upgrades would need a way to specify what
> package NVR they are upgrading (doable) and dependencies, requires, and
> obsoletes would need to be reviewed to ensure you don't wipe out a
> version you want installed.  Plus more.  Solvable and the same end
> result for users, just a different approach.
>

I would actually really like to see rpm's multiversioning capabilities
extended to support this.


-- 
真実はいつも一つ!/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-14 Thread David Cantrell
On 6/14/19 2:52 AM, Remi Collet wrote:
> Le 13/06/2019 à 20:31, Adam Samalik a écrit :
>> So, I'd like to discuss the libgit issue [1] [2] we're experiencing. With a
>> help of a few people, I've put together this post to get us on common
>> ground: https://communityblog.fedoraproject.org/modularity-vs-libgit/
>>
>> There are few ideas about solving the issue right now. But we might be able
>> to think about better ways to deal with similar issues long-term. Let's do
>> this!
> 
> IMHO, having library in modules is an error, this can only raise issues
> 
> Perhaps debian was right, and we should use a naming schema matching the
> library ABI, so including the soname
> 
>   libgit26-0.26.8
>   libgit27-0.27.8
>   libgit28-0.28.1
>   etc
> 
> Thanks to soname, library are perfect use case for parallel installation
> of multiple versions.

+1

We could go a step further and extend rpm and dnf to support multiple
versions of same named packages for installation.  This is doable but
not necessarily trivial.  Upgrades would need a way to specify what
package NVR they are upgrading (doable) and dependencies, requires, and
obsoletes would need to be reviewed to ensure you don't wipe out a
version you want installed.  Plus more.  Solvable and the same end
result for users, just a different approach.

> 
> And thanks to range version, this can be describe for dependencies.
> 
> BuildRequires: (pkgconfig(libgit) >=26 with pkgconfig(libgit) <= 27)
> 
> Yes, this will means more packages, and more reviews, but having a new
> review on a old library when soname change make sense.
> 
> Of course, old versions have to be retired when no more used
> 
> Modules are fine for applications.
> 
> 
> Remi
> 
> 
> P.S. yes compat_* name schema also works... but also consider it as a
> bit ugly...
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> 


-- 
David Cantrell 
Red Hat, Inc. | Boston, MA | EST5EDT
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-14 Thread Carl George
> So what is the point of modules then? If we want just multiple
> versions of a few applications, we can just have few repositories.

Add another repository for every alternate version (stream) doesn't scale.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-14 Thread Daniel Mach

Dne 14. 06. 19 v 6:23 Samuel Sieb napsal(a):
After reading the bug report and the discussions, I still don't 
understand why dnf is complaining about a conflict with packages 
(modules?) that are not installed and are not even trying to be 
installed.  Can someone explain that?


The dependency resolution is done on 2 levels.
RPM dependency resolution still works the same even with modules.

What's different on a modular system is a pre-processing step that 
resolves modules and filters out RPMs that do not match with 
enabled/default streams.


What makes it even more complicated is stream expansion:
https://docs.fedoraproject.org/en-US/modularity/architecture/stream-expansion/

Some modules get build several times against multiple dependency sets.
These dependency sets are captured as Context field in module metadata.
DNF has to resolve the contexts and pick the right ones based on enabled 
streams. This is done for *every* module version (not just the latest) 
in order to compute which RPMs will be available to the user and which 
not. And this causes the conflicts you were asking about.

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-14 Thread Igor Gnatenko
> Modules are fine for applications.

So what is the point of modules then? If we want just multiple
versions of a few applications, we can just have few repositories.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-14 Thread Fabio Valentini
On Fri, Jun 14, 2019, 09:42 Remi Collet  wrote:

> Le 13/06/2019 à 20:31, Adam Samalik a écrit :
> > So, I'd like to discuss the libgit issue [1] [2] we're experiencing.
> With a
> > help of a few people, I've put together this post to get us on common
> > ground: https://communityblog.fedoraproject.org/modularity-vs-libgit/
> >
> > There are few ideas about solving the issue right now. But we might be
> able
> > to think about better ways to deal with similar issues long-term. Let's
> do
> > this!
>
> IMHO, having library in modules is an error, this can only raise issues
>
> Perhaps debian was right, and we should use a naming schema matching the
> library ABI, so including the soname
>
> libgit26-0.26.8
> libgit27-0.27.8
> libgit28-0.28.1
> etc
>
> Thanks to soname, library are perfect use case for parallel installation
> of multiple versions.
>

I agree, this is a way better solution for libraries (where possible).


> And thanks to range version, this can be describe for dependencies.
>
> BuildRequires: (pkgconfig(libgit) >=26 with pkgconfig(libgit) <= 27)
>
> Yes, this will means more packages, and more reviews, but having a new
> review on a old library when soname change make sense.
>

Compat packages for existing libraries are already exempt from package
review, so that's not an issue.


> Of course, old versions have to be retired when no more used
>

This is already what's happening with versioned llvm, for example.


> Modules are fine for applications.
>
>
> Remi
>
>
> P.S. yes compat_* name schema also works... but also consider it as a
> bit ugly...
>

The Packaging Guidelines explicitly state that libfoo0, libfoo0.7 or
libgit2_0.27 patterns should be used for compat package names. The
compat-foo stuff is outdated.

Fabio

___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-14 Thread Vít Ondruch
There are two thing that stands out from the article:

* Multiple module names to parallel install

I hope you are not serious about it. We don't need modularity to do
things like that.

* Bundle libgit into the silver and bat modules

Not sure what you mean by this. I hope you mean that just as include the
libgit as another component of the module. So as a result, there would
be more libgit pakages available in Fedora.


But really. I think that currently modularity is doing way too much. If
the modules were just a way to get multiple versions of library into
repository, later let the DNF/RPM do its job to choose the right version
and fail if there should be multiple conflicting versions installed, you
deal with some default streams, trying to {,un,re}install components
forcefully. That is way beyond what should be done.


Also, I am personally happy that finally, you have to consider more
complex scenarios then just modulemd referencing sindle nodejs package,
which you basically used to design whole modularity. Welcome in the real
world.

Also note that we have similar discussions with plain RPM/YUM/DNF, we
are facing the same issue with (RH)SCLs and finally, you arrived on them
as well, although you have long claimed, that modularity will save the
world.


Vít



Dne 13. 06. 19 v 20:31 Adam Samalik napsal(a):
> So, I'd like to discuss the libgit issue [1] [2] we're experiencing.
> With a help of a few people, I've put together this post to get us on
> common
> ground: https://communityblog.fedoraproject.org/modularity-vs-libgit/
>
> There are few ideas about solving the issue right now. But we might be
> able to think about better ways to deal with similar issues long-term.
> Let's do this!
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1717117
> [2] https://pagure.io/fesco/issue/2146#comment-575852
> -- 
>
> Adam Šamalík
> ---
> Senior Software Engineer
> Red Hat
>
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-14 Thread Remi Collet

> IMHO, having library in modules is an error, this can only raise issues

Some examples, taken from RHEL-8

libJudy is part of the mariadb module ... (without -devel)
libssh2 is part of the virt module... (without -devel)
libzip is part of the php module...
etc

How are we going to manage this in EPEL ?



Remi
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-14 Thread Remi Collet
Le 13/06/2019 à 20:31, Adam Samalik a écrit :
> So, I'd like to discuss the libgit issue [1] [2] we're experiencing. With a
> help of a few people, I've put together this post to get us on common
> ground: https://communityblog.fedoraproject.org/modularity-vs-libgit/
> 
> There are few ideas about solving the issue right now. But we might be able
> to think about better ways to deal with similar issues long-term. Let's do
> this!

IMHO, having library in modules is an error, this can only raise issues

Perhaps debian was right, and we should use a naming schema matching the
library ABI, so including the soname

libgit26-0.26.8
libgit27-0.27.8
libgit28-0.28.1
etc

Thanks to soname, library are perfect use case for parallel installation
of multiple versions.

And thanks to range version, this can be describe for dependencies.

BuildRequires: (pkgconfig(libgit) >=26 with pkgconfig(libgit) <= 27)

Yes, this will means more packages, and more reviews, but having a new
review on a old library when soname change make sense.

Of course, old versions have to be retired when no more used

Modules are fine for applications.


Remi


P.S. yes compat_* name schema also works... but also consider it as a
bit ugly...
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-13 Thread Carl George
I think the best option is to create non-modular compat packages.  In my 
opinion, modularity makes sense for end user applications, but I'm not sure 
what benefits it has for libraries.  Libraries tend to work well as compat 
packages, so I implemented this in copr to try it out.

* https://copr.fedorainfracloud.org/coprs/carlwgeorge/parallel-libgit2/
* 
https://copr-be.cloud.fedoraproject.org/results/carlwgeorge/parallel-libgit2/fedora-rawhide-x86_64/00935643-libgit2_0.26/libgit2_0.26.spec
* 
https://copr-be.cloud.fedoraproject.org/results/carlwgeorge/parallel-libgit2/fedora-rawhide-x86_64/00935642-libgit2_0.27/libgit2_0.27.spec

This copr allows the following non-modular packages to be installed at the same 
time:

* Rawhide
* libgit2   0.28.2
* libgit2_0.27  0.27.8
* libgit2_0.26  0.26.8

* Fedora 30
* libgit2   0.27.8
* libgit2_0.26  0.26.8

These packages follow the current naming and conflict packaging guidelines.

* https://docs.fedoraproject.org/en-US/packaging-guidelines/Naming/#multiple
* 
https://docs.fedoraproject.org/en-US/packaging-guidelines/Conflicts/#_compat_package_conflicts

If I'm missing anything about the benefits of a modular libgit2 help me 
understand.  Given the current issues, this seems like a reasonable solution.  
If other agree, I'm happy to submit these compat packages for review.

Carl George
Rackspace
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-13 Thread Samuel Sieb
After reading the bug report and the discussions, I still don't 
understand why dnf is complaining about a conflict with packages 
(modules?) that are not installed and are not even trying to be 
installed.  Can someone explain that?

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Modularity vs. libgit

2019-06-13 Thread Igor Gnatenko
Hello Adam and everyone else,

I want to point out that there are 2 issues and not just one as
described in the article.

1) Default streams can't conflict
2) Switching dependencies on a module does not work

Both of the issues are solvable and should be handled by DNF, however
they are not. And I could not get any ETA on fixing that.

So the article talks about second issue. I'll comment on it first and
then I'll talk more about first issue.

# One stream with compatibility packages

This is not entirely doable. While library itself can be installed in
parallel, the devel package can not. So this would mean I would either
have to ship only one devel package which would not make sense or do
terrible hacks to install them in parallel (like mangling /usr/include
directory, pkg-config and such) and it won't work correctly. And after
all, what is the advantage of modularity in this case? I can do the
same in traditional Fedora.

# New streams of silver and bat

So we will have then silver:rolling, silver:rolling-new,
silver:rolling-very-new and such? Note that I simply can't support old
streams. They would need to go EOL as soon as I create new one. And we
decided not to do. Streams must be supported by people within active
Fedora release. So this is not solution to a problem at all.

# Bundle libgit into the silver and bat modules

Yes, I can link libgit2 statically everywhere. I don't even have to
include that package, I just have to ship static version of libgit2.
The problem here is that since in Fedora we don't have any automation
for rebuilding modules / packages, every time something changes in
libgit2, I would have manually go, check what to rebuild and do that
rebuilds.

# Potential solutions for the future

So we don't even look at solution named "Just Fix DNF"?


Now to the problem which was not described in the article.. In order
to describe, I'll start in short points what was before and what has
changed.

* Fedora 29 and 30 have non-modular libgit2-0.27
* Fedora 31 has had non-modular libgit2-0.27 as well, but last or this
week has upgraded to libgit2-0.28
* All Fedora versions ship 0.26, 0.27 and 0.28 streams of libgit2 as a modules
* All Fedora versions have default stream set to match non-modular
version (due to not having Ursa Prime ready)
* silver, bat, exa and more modules have default stream and used to
depend on libgit2:0.27 and have been rebuilt (most of them) to depend
on libgit2:0.28

Here I should introduce what "default stream" means for users. Users
should be able to just do "dnf install silver" and that should enable
default stream of module which provides package "silver". That means,
until you do installation, they can conflict. It might be not very
nice for users, but this is valid. So if we have default tokei:rolling
which depends on libgit2:0.27 and default libgit2:0.28, you should be
able to install either tokei which would install libgit2:0.27 or if
you do "dnf install libgit2", it would install 0.28 and you won't be
able to install tokei. What happens today is DNF just always complains
and does not allow you to install an operating system.

So simply untagging new builds of modules won't help, because then
modules will start depending back on libgit2:0.27 while rawhide has
already default for libgit2:0.28. Neither you can change a default
back to 0.27 because non-modular libgit2 has been upgraded and all
packages were rebuilt.

There are 2 solutions for this problem (not including fix of DNF):

1) Unset all defaults for Rust applications. This will solve problem
of not being to install an OS, but it won't solve issue with upgrades.
2) Untag new builds for Fedora 29 and 30, stop shipping updates for
Rust applications there and let them rot. For rawhide, I can rebuild
last few modules to start depending on libgit2:0.28, so rawhide will
be back on track.

First option is just horrible. Second one is okay-ish, but before this
way would be chosen I want to hear what is the ETA for fixing these
DNF bugs. I do not want Rust applications just rot in stable releases.

On Thu, Jun 13, 2019 at 9:22 PM Adam Samalik  wrote:
>
> So, I'd like to discuss the libgit issue [1] [2] we're experiencing. With a 
> help of a few people, I've put together this post to get us on common ground: 
> https://communityblog.fedoraproject.org/modularity-vs-libgit/
>
> There are few ideas about solving the issue right now. But we might be able 
> to think about better ways to deal with similar issues long-term. Let's do 
> this!
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1717117
> [2] https://pagure.io/fesco/issue/2146#comment-575852
> --
>
> Adam Šamalík
> ---
> Senior Software Engineer
> Red Hat
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.f

Modularity vs. libgit

2019-06-13 Thread Adam Samalik
So, I'd like to discuss the libgit issue [1] [2] we're experiencing. With a
help of a few people, I've put together this post to get us on common
ground: https://communityblog.fedoraproject.org/modularity-vs-libgit/

There are few ideas about solving the issue right now. But we might be able
to think about better ways to deal with similar issues long-term. Let's do
this!

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1717117
[2] https://pagure.io/fesco/issue/2146#comment-575852
-- 

Adam Šamalík
---
Senior Software Engineer
Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org