Re: Modularity and the system-upgrade path

[Petr Pisar]

On 2019-11-20, John M. Harris Jr  wrote:
> On Tuesday, November 19, 2019 3:52:27 AM MST Petr Pisar wrote:
>> If you start fidling with things in PATH, you have the problem of SCL. And
>> as you wrote, SCL is terrible. And that was the reason why we have
>> modularity: We do not want to relocate code to non-standard paths.
>
> I may be a bit confused here, but I thought Modularity was not a replacement 
> for SCLs? Clearly, it can't be, it doesn't provide even similar 
> functionality.. With SCLs, as annoying as they are, you do get parallel 
> installations, which Modularity cannot provide.
>
.
Start reading from "SCLs" keyword.

> If parallel availability, without parallel installation, is all you want, I 
> can show you how to do that with RPM right now, no Modularity required.
>
I would be happy for the parallel availability without Modularity. And it's
not a big deal on the installation part (except of ugly packages names).
The issue is the part when we build packages. One of the reasons why
Modularity is as it is is that RPM and Koji stated no interest in
accepting any changes. Therefore Modularity is a layer above them.

-- Petr
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[John M. Harris Jr]

On Tuesday, November 19, 2019 9:20:29 AM MST Kevin Kofler wrote:
> But the scripts do not need to care about the version of Perl you are 
> running, do they? It matters for compiled code, but why for Perl scripts? 
> Those can just run with the default version of Perl if they support it, or 
> with the shebang line changed to something like #!/usr/bin/perl5.30 if
> that's what they require.

There are certain edge cases where the version really does matter for scripts, 
but for most scripts you would be correct. I also agree with your solution for 
scripts that actually do require a specific version, however.

> The best way to deal with conflicts in PATH is to suffix the binaries, not 
> to move them. But that is only needed when it makes a difference for the
> end user which version they run. If the executable script "foo" does the
> exact same thing when run under Perl 5.28 or 5.30, then you need only one
> /usr/bin/foo set up to run against the distribution default Perl, the other
> one is redundant (which is the nice thing about parallel installation: you
> do not have to support running all the executables under a non-default
> Perl, only those that actually need it).

While that would work well in the Perl context, there are cases where it 
wouldn't work, for example there are several programs which hard-code paths 
which we would need to come up with an alternative path for and patch.

> I think that any model that has conflicts is not workable for the Fedora 
> user base. Desktops and small servers are not normally containerized, so 
> being able to install different applications without conflict is a non-
> negotiable requirement.

Agreed, in fact most workstations and servers with RHEL are not containerized 
either. Before RHEL 8, which was essentially just released, nothing even 
recommended containerizing RHEL.

-- 
John M. Harris, Jr.
Splentity

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[John M. Harris Jr]

On Tuesday, November 19, 2019 3:52:27 AM MST Petr Pisar wrote:
> If you start fidling with things in PATH, you have the problem of SCL. And
> as you wrote, SCL is terrible. And that was the reason why we have
> modularity: We do not want to relocate code to non-standard paths.

I may be a bit confused here, but I thought Modularity was not a replacement 
for SCLs? Clearly, it can't be, it doesn't provide even similar 
functionality.. With SCLs, as annoying as they are, you do get parallel 
installations, which Modularity cannot provide.

If parallel availability, without parallel installation, is all you want, I 
can show you how to do that with RPM right now, no Modularity required.

-- 
John M. Harris, Jr.
Splentity

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Kevin Kofler]

Petr Pisar wrote:
> That's nice theory that will never come true becaue it would require to
> make all Perl code parallel-installable. And Perl code is not only
> libraries as in the Python language. That's also myriad of Perl scripts
> that you want to have in PATH.

But the scripts do not need to care about the version of Perl you are 
running, do they? It matters for compiled code, but why for Perl scripts? 
Those can just run with the default version of Perl if they support it, or 
with the shebang line changed to something like #!/usr/bin/perl5.30 if 
that's what they require.

> If you start fidling with things in PATH, you have the problem of SCL. And
> as you wrote, SCL is terrible. And that was the reason why we have
> modularity: We do not want to relocate code to non-standard paths.

I agree that the SCL approach is not optimal, but letting the versions just 
conflict is much worse!

The best way to deal with conflicts in PATH is to suffix the binaries, not 
to move them. But that is only needed when it makes a difference for the end 
user which version they run. If the executable script "foo" does the exact 
same thing when run under Perl 5.28 or 5.30, then you need only one 
/usr/bin/foo set up to run against the distribution default Perl, the other 
one is redundant (which is the nice thing about parallel installation: you 
do not have to support running all the executables under a non-default Perl, 
only those that actually need it).

> I think it's inevitable that there will be conflicts and it's better to
> have them managable with a package manager (i.e. having default
> streams) rather crates few modules that silently overlay non-modular
> packages whe a user enables them.
> 
> The parallel installablity is nice, but it's way more difficult than
> parallel availability.

I think that any model that has conflicts is not workable for the Fedora 
user base. Desktops and small servers are not normally containerized, so 
being able to install different applications without conflict is a non-
negotiable requirement.

I see only 2 ways to provide a newer Perl for Fedora:
1. as a parallel-installable compatibility package, or
2. as a grouped official Bodhi update including a rebuild of all packages
   depending on the old Perl ABI
(and only the first one is suitable if you wish to provide an older Perl, 
because you should not be downgrading the system Perl).
Failing those, the only option is really:
3. just don't do it.

Providing a perl:5.30 module replacing the system Perl (and breaking 
everything in the distro depending on it) is essentially useless and does 
not provide much value over option 3.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Petr Pisar]

On 2019-11-19, Igor Gnatenko  wrote:
> Yes, but what you have described is basically to create 2 streams of
> perl-Sys-Virt module. Which is probably not what normal people want.

Having two different perl-Sys-Virt packages was requesed by
Daniel. That was not my choice.

> Creating module for one package is the worst idea ever.
>
Matter of opinion.

> Sure, bundling perl-Sys-Virt into the libvirt module would solve the
> problem, but then what's the point of modules? You will be building
> libvirt itself then multiple times due to the stream expansion.
> 
Then put perl-Sys-Virt into a separat module.

And please to do not top-post.

-- Petr
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Igor Gnatenko]

Yes, but what you have described is basically to create 2 streams of
perl-Sys-Virt module. Which is probably not what normal people want.
Creating module for one package is the worst idea ever.

Sure, bundling perl-Sys-Virt into the libvirt module would solve the
problem, but then what's the point of modules? You will be building
libvirt itself then multiple times due to the stream expansion.

On Tue, Nov 19, 2019 at 11:38 AM Petr Pisar  wrote:
>
> On 2019-11-15, Igor Gnatenko  wrote:
> > On Fri, Nov 15, 2019, 17:38 Petr Pisar  wrote:
> >> On 2019-11-15, Daniel P  Berrang=C3=A9  wrote:
> >> >
> >> > Consider if we move the virtualization stack (QEMU & Libvirt) into a
> >> > module with two streams, one libvirt 5.8.0 and one libvirt 6.1.0.
> >> >
> >> > Now we want to build Perl bindings for libvirt. We'll need the
> >> > corresponding version of perl-Sys-Virt either 5.8.0 or 6.1.0,
> >> > built for each virt module stream, but also built for each Perl
> >> > module stream 5.26 / 5.30. eg the combinatorial expansion
> >> >
> >> >  -   perl-Sys-Virt 5.8.0   with libvirt 5.9.0 with perl 5.26
> >> >  -   perl-Sys-Virt 5.8.0   with libvirt 5.9.0 with perl 5.30
> >> >  -   perl-Sys-Virt 6.1.0   with libvirt 6.1.0 with perl 5.26
> >> >  -   perl-Sys-Virt 6.1.0   with libvirt 6.1.0 with perl 5.30
> [...]
> > The problem described by Daniel was that Perl module should be different
> > version when building against 5.x libvirt.
> >
> >> Example: Let's say you have libvirt module with 5.8.0 and 6.1.0 streams
> >> and perl module with 5.26 and 5.30 streams. If you add perl-Sys-Virt
> >> into a new module, you write a modulemd file for it like this:
> >>
> >> - buildrequiers:
> >> libvirt: [5.8.0, 6.1.0]
> >> perl: [5.26, 5.30]
> >> platform: [f32]
> >>   requires:
> >> libvirt: [5.8.0, 6.1.0]
> >> perl: [5.26, 5.30]
> >> platform: [f32]
> >>
> You are right. Then either you put perl-Sys-Virt 5.8.0 into
> a perl-Sys-Virt:5.8.0 stream with this dependency specification:
>
> - buildrequiers:
> libvirt: [5.9.0]
> perl: [5.26, 5.30]
> platform: [f32]
>   requires:
> libvirt: [5.9.0]
> perl: [5.26, 5.30]
> platform: [f32]
>
> and perl-Sys-Virt 6.1.0 package into perl-Sys-Virt:6.1.0 stream with:
>
> - buildrequiers:
> libvirt: [6.1.0]
> perl: [5.26, 5.30]
> platform: [f32]
>   requires:
> libvirt: [6.1.0]
> perl: [5.26, 5.30]
> platform: [f32]
>
> Or you put perl-Sys-Virt package into libvirt module and for
> libvirt:5.9.0 stream you write:
>
> - buildrequiers:
> perl: [5.26, 5.30]
> platform: [f32]
>   requires:
> perl: [5.26, 5.30]
> platform: [f32]
>
> and for libvirt:6.1.0 stream you do the same.
>
> What approach you want to choose probably depends on compatibility
> among perl-Sys-Virt package versions and among libvirt versions. And how
> often they are released.
>
> I.e. if you can rebase perl-Sys-Virt inside libvirt stream because
> perl-Sys-Virt does not break ABI, then it makes sense to keep it inside
> libvirt module. That's because public ABI of a module should not change
> inside a stream.
>
> You can also consider how expensive is to build, test and deliver the
> libvirt module. If e.g. building perl-Sys-Virt were much quicker than
> building libvirt, and there were plenty of perl streams, then it would
> make sense to move perl-Sys-Virt package into its own module.
>
> I think it's a similar problem as to when bundle all dependencies into
> one package and when to aim for splitting it into muliple independent
> packages.
>
> -- Petr
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Petr Pisar]

On 2019-11-16, Kevin Kofler  wrote:
> Petr Pisar wrote:
>> With your proposal Bugzilla packager would have to package Bugzilla
>> twice: as a normal package for default Perl 5.26 and as a module for Perl
>> 5.30. Then a user would have hard time to select the right combinations of
>> Perl and Bugzilla. It would double fork work pacakgers and and make
>> the system more dificult for users.
>
> The Bugzilla rebuild for the non-default Perl actually belongs IN the Perl 
> module. Otherwise, enabling the non-default stream for the Perl module will 
> break the user's Bugzilla and force them to manually enable the 
> corresponding non-default stream for the Bugzilla module. Plus, since there 
> are many Perl applications, having a module for each of them (each tracking 
> Perl's module streams) just does not scale.
> 
Adding Bugzilla in Perl module does not scale either. You have many Perl
applications and you should put them all into the Perl module. As
a result you would have a Perl module containing all Perl applications.
We now more than 3000 such packages. You can imagine that you were never
be eable to build such a giant module because there is always a package
that fails to build. Also whenever you want to provide a new
incompatible application you would have to fork the giant Perl module.
Do you want to have perl:5.30-with-Bugzilla-5.1.2-with-slic3r-1.3.0-...?

I believe that better approach is to meld modularity into RPM. So that
you can have plenty of small independent packages aware of parallel
availability.

> But what this example really shows is that it is a horrible idea to have a 
> Perl module to begin with. The non-default Perl needs to be packaged as a 
> parallel-installable compatibility package (or as an SCL, but that opens its 
> own can of worms) instead. You cannot just replace a language interpreter 
> (especially not one as widely used as Perl) with a different version. (As 
> you pointed out yourself, that breaks even fedpkg. Even though fedpkg itself 
> is not even written in Perl!) The parallel-installable approach is also the 
> only reasonable way to support applications that require a non-default 
> version of Perl, without conflicting with the rest of the distribution.
>
That's nice theory that will never come true becaue it would require to
make all Perl code parallel-installable. And Perl code is not only
libraries as in the Python language. That's also myriad of Perl scripts
that you want to have in PATH. If you start fidling with things in PATH,
you have the problem of SCL. And as you wrote, SCL is terrible. And that
was the reason why we have modularity: We do not want to relocate code
to non-standard paths.

I think it's inevitable that there will be conflicts and it's better to
have them managable with a package manager (i.e. having default
streams) rather crates few modules that silently overlay non-modular
packages whe a user enables them.

The parallel installablity is nice, but it's way more difficult than
parallel availability.

-- Petr
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Petr Pisar]

On 2019-11-15, Igor Gnatenko  wrote:
> On Fri, Nov 15, 2019, 17:38 Petr Pisar  wrote:
>> On 2019-11-15, Daniel P  Berrang=C3=A9  wrote:
>> >
>> > Consider if we move the virtualization stack (QEMU & Libvirt) into a
>> > module with two streams, one libvirt 5.8.0 and one libvirt 6.1.0.
>> >
>> > Now we want to build Perl bindings for libvirt. We'll need the
>> > corresponding version of perl-Sys-Virt either 5.8.0 or 6.1.0,
>> > built for each virt module stream, but also built for each Perl
>> > module stream 5.26 / 5.30. eg the combinatorial expansion
>> >
>> >  -   perl-Sys-Virt 5.8.0   with libvirt 5.9.0 with perl 5.26
>> >  -   perl-Sys-Virt 5.8.0   with libvirt 5.9.0 with perl 5.30
>> >  -   perl-Sys-Virt 6.1.0   with libvirt 6.1.0 with perl 5.26
>> >  -   perl-Sys-Virt 6.1.0   with libvirt 6.1.0 with perl 5.30
[...]
> The problem described by Daniel was that Perl module should be different
> version when building against 5.x libvirt.
>
>> Example: Let's say you have libvirt module with 5.8.0 and 6.1.0 streams
>> and perl module with 5.26 and 5.30 streams. If you add perl-Sys-Virt
>> into a new module, you write a modulemd file for it like this:
>>
>> - buildrequiers:
>> libvirt: [5.8.0, 6.1.0]
>> perl: [5.26, 5.30]
>> platform: [f32]
>>   requires:
>> libvirt: [5.8.0, 6.1.0]
>> perl: [5.26, 5.30]
>> platform: [f32]
>>
You are right. Then either you put perl-Sys-Virt 5.8.0 into
a perl-Sys-Virt:5.8.0 stream with this dependency specification:

- buildrequiers:
libvirt: [5.9.0]
perl: [5.26, 5.30]
platform: [f32]
  requires:
libvirt: [5.9.0]
perl: [5.26, 5.30]
platform: [f32]

and perl-Sys-Virt 6.1.0 package into perl-Sys-Virt:6.1.0 stream with:

- buildrequiers:
libvirt: [6.1.0]
perl: [5.26, 5.30]
platform: [f32]
  requires:
libvirt: [6.1.0]
perl: [5.26, 5.30]
platform: [f32]

Or you put perl-Sys-Virt package into libvirt module and for
libvirt:5.9.0 stream you write:

- buildrequiers:
perl: [5.26, 5.30]
platform: [f32]
  requires:
perl: [5.26, 5.30]
platform: [f32]

and for libvirt:6.1.0 stream you do the same.

What approach you want to choose probably depends on compatibility
among perl-Sys-Virt package versions and among libvirt versions. And how
often they are released.

I.e. if you can rebase perl-Sys-Virt inside libvirt stream because
perl-Sys-Virt does not break ABI, then it makes sense to keep it inside
libvirt module. That's because public ABI of a module should not change
inside a stream.

You can also consider how expensive is to build, test and deliver the
libvirt module. If e.g. building perl-Sys-Virt were much quicker than
building libvirt, and there were plenty of perl streams, then it would
make sense to move perl-Sys-Virt package into its own module.

I think it's a similar problem as to when bundle all dependencies into
one package and when to aim for splitting it into muliple independent
packages.

-- Petr
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Petr Pisar]

On 2019-11-15, Przemek Klosowski via devel  
wrote:
> Of course in practice the combinatorial behavior only happens within the 
> subsets of software that depend on each other, but, nevertheless, it 
> seems to me that this means that we have to control and limit the number 
> of interdependent modules drastically, like to single digits.
>
When it matters, maintainers can limit the number of combinations. E.g.
you can restrict to 2 combinations like this:

- buildrequiers:
libvirt: [5.8.0]
perl: [5.26]
platform: [f32]
  requires:
libvirt: [5.8.0]
perl: [5.26]
platform: [f32]
- buildrequiers:
libvirt: [6.1.0]
perl: [5.30]
platform: [f32]
  requires:
libvirt: [6.1.0]
perl: [5.30]
platform: [f32]

But don't forget that if a built module can actually work with many
streams at run-time, you cam simplify it like this:

- buildrequiers:
libvirt: [5.8.0]
perl: [5.26, 5.30]
platform: [f32]
  requires:
libvirt: [5.8.0, 6.1.0]
perl: [5.26, 5.30]
platform: [f32]

This declares that you want to make two builds and each of the builds
will be compatible with both libvirt streams. This is what Java modules
often do.

> we haven't found the right abstraction for dealing with software
> versioning yet.
>
I'm pessimistic. Fedora as any binary distribution distributes binaries.
ABI changes usually proliferate quicker than API incompatibilities.
That's why e.g. Gentoo does not have this issue because there you simply
put "spec files" for multiple versions into a repository and the exact
binary combination is formed at installation time on a user's machine.
There is missing "Koji" in the the distribution chain.

-- Petr
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Nicolas Mailhot via devel]

Le samedi 16 novembre 2019 à 19:05 +0100, clime a écrit :
> On Sat, 16 Nov 2019 at 18:54, Nicolas Mailhot via devel
>  wrote:
> > Le samedi 16 novembre 2019 à 18:42 +0100, clime a écrit :
> > > On Sat, 16 Nov 2019 at 08:38, Nicolas Mailhot via devel
> > >  wrote:
> > > > Le samedi 16 novembre 2019 à 03:38 +0100, clime a écrit :
> > > > > > A true solution would be blending modularity into RPM.
> > > > > > At build time as well as at installation time.
> > > > > 
> > > > > I agree this would be the best. Basically, final
> > > > > product of a module build should be an rpm. modulemd
> > > > > file should be kind of a meta-spec file
> > > > 
> > > > There should be no need for a modulemd *at* *all*.
> > > 
> > > modulemd + related infrastructure gives you distributed building,
> > > which is cool if you want to build a "solution" i.e. multiple
> > > software packages all combined to serve a particular use-case.
> > 
> > Yes it is wickedly cool as a distributed building solution.
> > 
> > It is not cool *at* all* as a replacement for spec declarations.
> > Just
> > put the correct variables in the spec files themselves, and have
> > the
> > distributed building solutions set them during  builds (as is done
> > for
> > dist)
> 
> Yes, but the point is, the product of the distributed build should be
> a single
> rpm so that you don't need to handle two kinds of objects during
> installation time and inter-dependencies between them. 

Really, this is the kind of technical choice that made modules fail in
practical terms. Just generate rpms with no magic except a module
marker. If you build n versions of a rpm, generate n rpms. Autoreqs
will register what you built against (and if they do not allow to
disambiguate different versions of the same object they should be
fixed).

Choosing to source a particular rpm in a particular module should not
be more than a user hint. The packages themselves, should just be
packages as usual.

Mass rebuilds are cool because the user does not need to know that the
resulting packages were built by a single all-encompasing build command
(usually, several releng iterations as problems are found and fixed).

Modules, has they exist today, leak build command grouping into install
grouping. That's why the install part of modules is breaking right and
left.

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[clime]

On Sat, 16 Nov 2019 at 18:54, Nicolas Mailhot via devel
 wrote:
>
> Le samedi 16 novembre 2019 à 18:42 +0100, clime a écrit :
> > On Sat, 16 Nov 2019 at 08:38, Nicolas Mailhot via devel
> >  wrote:
> > > Le samedi 16 novembre 2019 à 03:38 +0100, clime a écrit :
> > > > > A true solution would be blending modularity into RPM.
> > > > > At build time as well as at installation time.
> > > >
> > > > I agree this would be the best. Basically, final
> > > > product of a module build should be an rpm. modulemd
> > > > file should be kind of a meta-spec file
> > >
> > > There should be no need for a modulemd *at* *all*.
> >
> > modulemd + related infrastructure gives you distributed building,
> > which is cool if you want to build a "solution" i.e. multiple
> > software packages all combined to serve a particular use-case.
>
> Yes it is wickedly cool as a distributed building solution.
>
> It is not cool *at* all* as a replacement for spec declarations. Just
> put the correct variables in the spec files themselves, and have the
> distributed building solutions set them during  builds (as is done for
> dist)

Yes, but the point is, the product of the distributed build should be a single
rpm so that you don't need to handle two kinds of objects during installation
time and inter-dependencies between them. So there should be a single
spec file generated partially automatically (i.e. by collecting what sources
were built and putting them into the corresponding Source: statements)
and partially manually (i.e. scriplets need to be pulled from somewhere).

>
> Regards
>
> --
> Nicolas Mailhot
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Nicolas Mailhot via devel]

Le samedi 16 novembre 2019 à 18:42 +0100, clime a écrit :
> On Sat, 16 Nov 2019 at 08:38, Nicolas Mailhot via devel
>  wrote:
> > Le samedi 16 novembre 2019 à 03:38 +0100, clime a écrit :
> > > > A true solution would be blending modularity into RPM.
> > > > At build time as well as at installation time.
> > > 
> > > I agree this would be the best. Basically, final
> > > product of a module build should be an rpm. modulemd
> > > file should be kind of a meta-spec file
> > 
> > There should be no need for a modulemd *at* *all*.
> 
> modulemd + related infrastructure gives you distributed building,
> which is cool if you want to build a "solution" i.e. multiple
> software packages all combined to serve a particular use-case.

Yes it is wickedly cool as a distributed building solution.

It is not cool *at* all* as a replacement for spec declarations. Just
put the correct variables in the spec files themselves, and have the
distributed building solutions set them during  builds (as is done for
dist)

Regards

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[clime]

On Sat, 16 Nov 2019 at 08:38, Nicolas Mailhot via devel
 wrote:
>
> Le samedi 16 novembre 2019 à 03:38 +0100, clime a écrit :
> > > A true solution would be blending modularity into RPM.
> > > At build time as well as at installation time.
> >
> > I agree this would be the best. Basically, final
> > product of a module build should be an rpm. modulemd
> > file should be kind of a meta-spec file
>
> There should be no need for a modulemd *at* *all*.

modulemd + related infrastructure gives you distributed building,
which is cool if you want to build a "solution" i.e. multiple software
packages all combined to serve a particular use-case. Also passing
compile time option to tweak the given build is allowed by modulemd.
You don't get something like that when building a spec file and i think
it would be hard to achieve.

>
> Specifying a module stream target for a build should be just an rpm
> variable, in ~/.config/rpm/macros or passed on the cli to rpmbuild,
> etc.
>
> Exactly like we do with dist. We managed to handle multiple
> distribution releases with dist, even if it was not a core rpm concept,
> without needing to change rpm at all. And it works. And it didn't cause
> half the havoc of modules.
>
> Sure, do some rpm fixing if necessary so the result feels less like a
> kludge than %dist. But, don’t rely on an external framework to do
> things for you instead of doing the necessary work (if any) at the
> component format level.
>
> Module upgrade and conflict resolution strategies should be just dnf
> modules over this basic rpm format. So we can have change the strategy
> over time without changing the packages and specs themselves.
>
> And *then*, you can build all kinds of templating management frameworks
> over those basic format changes. In fact people being people they will
> build multiple frameworks, in all kind of languages, and argue
> endlessly which one is best. That won’t matter because the low-level
> module info and format will exist directly in rpm.
>
> Modules started with the end-user management framework (porcelain)
> part, and got lost somewhere trying to decide how to map it to low
> level concepts. That, does not work. Start from fundations before
> arguing about the roof decorations.
>
> Regards,
> --
> Nicolas Mailhot
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Nicolas Mailhot via devel]

Le samedi 16 novembre 2019 à 09:35 +0100, Lukas Ruzicka a écrit :
> 
> 
> > Either the strategy should be:
> > 
> > "We offer alternate Perl versions for containers etc. they conflict
> > with the 
> > default Perl version and with the non-modular apps. That is known
> > and accepted."

That won't work.

You *can* ship modular components as alternatives

You *can* *not* ship modular components that directly compete with non-
modular as defaults, because the maintainer of non modular will have
made the effort to integrate with the rest of the distro, and not only
the maintainer of the modular content won't have made this effort, but
he will offload the integration problems *he* created to the maintainer
of the default non modular version

The root reason of the current mess is that modular was allowed to
override things without owning the problems that created.

Modular should *not* override anything by default in dep resolution.
Dep resolution *may* use modular preferences as hints, but those hints
should be weak at best, and be ignored by the dep resolver as soon as
they conflict with dep graph resolution.

And yes that means maintainers of modular things will get the rug
pulled under them whenever the default stream changes in incompatible
ways. Tough luck. The only reliable way we have to coordinate Fedora
activity is this default stream.

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Lukas Ruzicka]

Either the strategy should be:
>
> "We offer alternate Perl versions for containers etc. they conflict with
> the
> default Perl version and with the non-modular apps. That is known and
> accepted."
>
> Or the strategy should be:
>
> "We build all our Perl applications for all our Perl versions, so users
> who
> choose their Perl stream can still keep their applications from the
> distribution."
>
>
Exactly. While I am thinking that the *first strategy is easily achievable*,
even with what we already have,
the *second strategy is very complicated* to achieve, because we cannot
predict what applications
users want to install and in which versions. They all would have to work in
Fedora, otherwise the distro
does not make sense any more. Let me explain.

If I know that installing an alternative version of Perl could break Perl
bindings to other applications, I can
create a container to use that alternative version of Perl and be happy,
having actually the standard Perl in the
system and another version in the container, or I just install a system
with that, because I only need
to have one purpose operating system, such as LAMP or other servers. That's
fine.

For desktop users, however, this is not good because you place limitations
on them. While I believe it is fairly ok
to build a server solution around containers (or even virtual machines),
this is overly complicated for Desktop.
I do not understand why we would like to make Desktop complicated, when the
majority of Red Hat use Fedora as
their desktop solution. Also, there are spins that basically are
Workstations (or other desktops) that have certain
packages pre-installed and we expect them to work flawlessly.

The questions is: *With modularity ... can we make sure that everything
works with everything as it does nowadays?*

I believe that having non-modular defaults will make sure the distro works
in its entirety, while having alternative versions
in modules will help developers and sysadmins to install what they want and
need, if it is not the default. For me, this is a win win situation.
I understand that it is more work for the packager, but it is more
convenient for the users and we should think about the users
in the first place.

I have been following this discussion since it started and all I am getting
is "We are having issues, but we are working on them.",
but nobody has ever explained why it is bad to use Miro's approach.



-- 

Lukáš Růžička

FEDORA QE, RHCE

Red Hat



Purkyňova 115

612 45 Brno - Královo Pole

lruzi...@redhat.com
TRIED AND PERSONALLY TESTED, ERGO TRUSTED. 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Nicolas Mailhot via devel]

Le samedi 16 novembre 2019 à 08:37 +0100, Nicolas Mailhot a écrit :
> 
> Sure, do some rpm fixing if necessary so the result feels less like a
> kludge than %dist. But, don’t rely on an external framework to do
> things for you instead of doing the necessary work (if any) at the
> component format level.

Hell, there’s no even any need to change rpm itself, Group exists and
has been liberated from previous use..

Just shove the module name in Group, write all the necessary macros and
templates to decline %{group} info in necessary parts of the spec,
write the necessary dnf plugins to do smart things with Group info.

And then when that is shown working (as in, upgrade conflict resolution
works), and the correct way to do modules is finaly understood, the
behaviour of the macros and dnf plugin can be streamlined and merged
and hardcoded in rpm and dnf cores.

(which may never happen, as dist showed, but that's not a problem with
rpm the application, that's a problem with Red Hat and rpm the project
not investing in cleaning up technical debt)

Regards,

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Nicolas Mailhot via devel]

Le samedi 16 novembre 2019 à 03:38 +0100, clime a écrit :
> > A true solution would be blending modularity into RPM.
> > At build time as well as at installation time.
> 
> I agree this would be the best. Basically, final
> product of a module build should be an rpm. modulemd
> file should be kind of a meta-spec file

There should be no need for a modulemd *at* *all*.

Specifying a module stream target for a build should be just an rpm
variable, in ~/.config/rpm/macros or passed on the cli to rpmbuild,
etc.

Exactly like we do with dist. We managed to handle multiple
distribution releases with dist, even if it was not a core rpm concept,
without needing to change rpm at all. And it works. And it didn't cause
half the havoc of modules.

Sure, do some rpm fixing if necessary so the result feels less like a
kludge than %dist. But, don’t rely on an external framework to do
things for you instead of doing the necessary work (if any) at the
component format level.

Module upgrade and conflict resolution strategies should be just dnf
modules over this basic rpm format. So we can have change the strategy
over time without changing the packages and specs themselves.

And *then*, you can build all kinds of templating management frameworks
over those basic format changes. In fact people being people they will
build multiple frameworks, in all kind of languages, and argue
endlessly which one is best. That won’t matter because the low-level
module info and format will exist directly in rpm.

Modules started with the end-user management framework (porcelain)
part, and got lost somewhere trying to decide how to map it to low
level concepts. That, does not work. Start from fundations before
arguing about the roof decorations.

Regards,
-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[clime]

> A true solution would be blending modularity into RPM.
> At build time as well as at installation time.

I agree this would be the best. Basically, final
product of a module build should be an rpm. modulemd
file should be kind of a meta-spec file which can be built
distributively and that contains parts which are then inserted
into the standard resulting spec file, which can contain
multiple applications or a just single one. "Stream" can
be a new rpm property that could be used by user as
an extra-specifier during installation. In other words,
modularity should be all about build-time. In the end,
standard rpm should be the result. That way, there
are no collisions between modular and non-modular
on the user side because everything is rpm in the end
and only the way the rpm was produced differs.

On Fri, 15 Nov 2019 at 19:11, Petr Pisar  wrote:
>
> On 2019-11-15, Miro Hrončok  wrote:
> > On 15. 11. 19 16:11, Petr Pisar wrote:
> >> On 2019-11-15, Miro Hrončok  wrote:
> >>> On 15. 11. 19 14:32, Petr Pisar wrote:
>  On 2019-10-04, Miro Hrončok  wrote:
> > Wouldn't it be easier if the "default stream" would just behave like
> > a regular package?
> >
> > I can think of two solutions of that:
> >
> > 1. (drastic for modular maintainers)
> >
> > We keep miantaining the default versions of things as ursine packages.
> > We only modularize alternate versions.
> >
>  Big con:
> 
>  That effectively bans modules with multiple dependencies where at least
>  one is a default version.
> 
>  Example: I have Perl 5.26 as a default version. I have Perl 5.30 as an
>  laternative version. Now I want to package Bugzilla that's written in
>  Perl. How do you package Bugzilla so that it works with Perl 5.26 as
>  well as with Perl 5.30?
> >>>
> >>> I don't understand why would the user care about the Perl version when
> >>> they want Bugzilla. How is Bugzilla different form e.g. Slic3r (app
> >>> that happens to be written in Perl)? Do we want to modularize all such
> >>> apps to solve the "no parallel instability" feature?
> >>>
> >> I don't know. Ask the user why he needs a different Perl version than
> >> the default one.  Maybe he has some other applications that work only
> >> with that particular version.
> >
> > What I was implying is that I don't understand why the user of Buzgilla 
> > wants
> > different Perl version to run it. I was not implying that users don't want
> > various Perl versions generally.
> >
> If the user is interested only in Bugzilla and nothing else, then of
> course he does not care about Perl version. Unfortunatelly people run
> more applications on their systems and then they have multiple
> requirements that can conflict each to other.
>
> >> If you believe that users do  not care about a version of software they
> >> use, then we can drop out modularity, and all Fedora releases and
> >> deliver only Rawhide. Or we can stop integrating new versions of
> >> software and deliver Fedora 32 and nothing else forever.
> >
> > I believe that the purpose of a distribution is to cerate and
> > integrated environment, where we simply make sure that Bugzilla works
> > and runs on a Perl version we support. And we move forward and
> > integrate with newer Perl versions.
> >
> I understand. I also don't care about versions until my system is
> compatible and supported. But the problem emerges when you start to care
> because you need a new feature or postpone an upgrade because the new
> version is broken for you.
>
> > Note that I don't necessarily mean that the use case doesn't exist,
> > I just say I don't really get it. And why is Bugzilla any different
> > that all other Perl applications.
> >
> Bugzilla is not any different. It was only an example.
>
> > Either the strategy should be:
> >
> > "We offer alternate Perl versions for containers etc. they conflict
> > with the default Perl version and with the non-modular apps. That is
> > known and accepted."
> >
> > Or the strategy should be:
> >
> > "We build all our Perl applications for all our Perl versions, so
> > users who choose their Perl stream can still keep their applications
> > from the distribution."
> >
> > I fail to see what are we trying to achieve here exactly.
>
> I would love this second option, but with current modularity it's not
> feasable. Not because of the juvenile defects we have now (like
> switching streams on distribution upgrades) but because it would require
> a module for each package. And current implementation does not scale so
> well and cannot describe all needed relations we have readilly available
> on an RPM level. A true solution would be blending modularity into RPM.
> At build time as well as at installation time.
>
> So the first option is more realistic. You correctly write "alternate
> [...] versions [...] conflict [...] with the non-modular apps".
>
> However, my intuition says that nobody will use the alternative versions
> 

Re: Modularity and the system-upgrade path

[Kevin Kofler]

Petr Pisar wrote:
> With your proposal Bugzilla packager would have to package Bugzilla
> twice: as a normal package for default Perl 5.26 and as a module for Perl
> 5.30. Then a user would have hard time to select the right combinations of
> Perl and Bugzilla. It would double fork work pacakgers and and make
> the system more dificult for users.

The Bugzilla rebuild for the non-default Perl actually belongs IN the Perl 
module. Otherwise, enabling the non-default stream for the Perl module will 
break the user's Bugzilla and force them to manually enable the 
corresponding non-default stream for the Bugzilla module. Plus, since there 
are many Perl applications, having a module for each of them (each tracking 
Perl's module streams) just does not scale.

But what this example really shows is that it is a horrible idea to have a 
Perl module to begin with. The non-default Perl needs to be packaged as a 
parallel-installable compatibility package (or as an SCL, but that opens its 
own can of worms) instead. You cannot just replace a language interpreter 
(especially not one as widely used as Perl) with a different version. (As 
you pointed out yourself, that breaks even fedpkg. Even though fedpkg itself 
is not even written in Perl!) The parallel-installable approach is also the 
only reasonable way to support applications that require a non-default 
version of Perl, without conflicting with the rest of the distribution.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Kevin Kofler]

Przemek Klosowski via devel wrote:
> so for one module with two versions, we will have 2 builds, for 2
> modules with two versions we'll have four builds, and in general for N
> modules with M versions on average, we will have N^M builds?

M^N actually.

1 module with M versions = M builds
2 modules with M versions each = M*M=M^2 builds
3 modules with M versions each = M*M*M=M^3 builds
…
n modules with M versions each = M^N builds

> This is a textbook combinatorial explosion:  100 modules with average 3
> versions each is a million builds and tests

It is actually 3^100 > 5*10^47 builds. That's more than
500 000 000 000 000 000 000 000 000 000 000 000 000 000 million builds, not 
1 million.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Miro Hrončok]

On 15. 11. 19 19:10, Petr Pisar wrote:

Now you can think another modularity fatatic who wants to modularize
everything and have all modules in multiple versions. Not at all.


Don't worry, I don't consider anybody a modular fanatic. Yet anyway.

Thanks for you answer, it has been valuable to me and I hope that it was 
valuable for other as well.


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Petr Pisar]

On 2019-11-15, Miro Hrončok  wrote:
> On 15. 11. 19 16:11, Petr Pisar wrote:
>> On 2019-11-15, Miro Hrončok  wrote:
>>> On 15. 11. 19 14:32, Petr Pisar wrote:
 On 2019-10-04, Miro Hrončok  wrote:
> Wouldn't it be easier if the "default stream" would just behave like
> a regular package?
>
> I can think of two solutions of that:
>
> 1. (drastic for modular maintainers)
>
> We keep miantaining the default versions of things as ursine packages.
> We only modularize alternate versions.
>
 Big con:

 That effectively bans modules with multiple dependencies where at least
 one is a default version.

 Example: I have Perl 5.26 as a default version. I have Perl 5.30 as an
 laternative version. Now I want to package Bugzilla that's written in
 Perl. How do you package Bugzilla so that it works with Perl 5.26 as
 well as with Perl 5.30?
>>>
>>> I don't understand why would the user care about the Perl version when
>>> they want Bugzilla. How is Bugzilla different form e.g. Slic3r (app
>>> that happens to be written in Perl)? Do we want to modularize all such
>>> apps to solve the "no parallel instability" feature?
>>>
>> I don't know. Ask the user why he needs a different Perl version than
>> the default one.  Maybe he has some other applications that work only
>> with that particular version.
>
> What I was implying is that I don't understand why the user of Buzgilla wants 
> different Perl version to run it. I was not implying that users don't want 
> various Perl versions generally.
>
If the user is interested only in Bugzilla and nothing else, then of
course he does not care about Perl version. Unfortunatelly people run
more applications on their systems and then they have multiple
requirements that can conflict each to other.

>> If you believe that users do  not care about a version of software they
>> use, then we can drop out modularity, and all Fedora releases and
>> deliver only Rawhide. Or we can stop integrating new versions of
>> software and deliver Fedora 32 and nothing else forever.
>
> I believe that the purpose of a distribution is to cerate and
> integrated environment, where we simply make sure that Bugzilla works
> and runs on a Perl version we support. And we move forward and
> integrate with newer Perl versions.
>
I understand. I also don't care about versions until my system is
compatible and supported. But the problem emerges when you start to care
because you need a new feature or postpone an upgrade because the new
version is broken for you.

> Note that I don't necessarily mean that the use case doesn't exist,
> I just say I don't really get it. And why is Bugzilla any different
> that all other Perl applications.
>
Bugzilla is not any different. It was only an example.

> Either the strategy should be:
>
> "We offer alternate Perl versions for containers etc. they conflict
> with the default Perl version and with the non-modular apps. That is
> known and accepted."
>
> Or the strategy should be:
>
> "We build all our Perl applications for all our Perl versions, so
> users who choose their Perl stream can still keep their applications
> from the distribution."
>
> I fail to see what are we trying to achieve here exactly.

I would love this second option, but with current modularity it's not
feasable. Not because of the juvenile defects we have now (like
switching streams on distribution upgrades) but because it would require
a module for each package. And current implementation does not scale so
well and cannot describe all needed relations we have readilly available
on an RPM level. A true solution would be blending modularity into RPM.
At build time as well as at installation time.

So the first option is more realistic. You correctly write "alternate
[...] versions [...] conflict [...] with the non-modular apps".

However, my intuition says that nobody will use the alternative versions
for exactly this reason. And I think I'm right. Look at me. I maintain
Perl modules but I don't use them. I cannot because I would have to
uninstall all the other Perl packages from my system. And not only Perl
packages. fedpkg transitively depends on non-modular Perl. Anybody who
wants a different Perl cannot be a Fedora packager.

Therefore I think it's desirable to modularize applications. Because
that way we diminish the conflicts and that increases value of the
distribution. Look how few modules we have now in Fedora.

Now you can think another modularity fatatic who wants to modularize
everything and have all modules in multiple versions. Not at all.
I believe most of the modules will exist only in one stream. But
the reason why we need to modularize everything is to actually enable
multiple stream for the "few" modules that actually can take benefits
from the multiple streams. Because once everything is a module, it's
trivial to add a new stream to a module in a middle of the dependency
tree. It's trivial to test the 

Re: Modularity and the system-upgrade path

[Przemek Klosowski via devel]

On 11/15/19 11:27 AM, Petr Pisar wrote:

No. Modularity solves this combination problem with "stream expansion".
Sources for such module exists only once, you submit them for building
with fedpkg only once, but a build systems computes all combinations
(this the stream expansion) and schedules a build for each of the
combination. That will result in multiple module builds with the same
module name, stream, version, but differing with a special
discriminator called "context".


so for one module with two versions, we will have 2 builds, for 2 
modules with two versions we'll have four builds, and in general for N 
modules with M versions on average, we will have N^M builds? This is a 
textbook combinatorial explosion:  100 modules with average 3 versions 
each is a million builds and tests, with million resulting versions to 
be picked from.


Of course in practice the combinatorial behavior only happens within the 
subsets of software that depend on each other, but, nevertheless, it 
seems to me that this means that we have to control and limit the number 
of interdependent modules drastically, like to single digits.


BTW, it always bothered me that in some sense the prime case for modules 
is the kernel---but the kernel has always been treated specially and is 
not being subsumed into modules. I think that is because we are thinking 
about the whole thing wrong; we haven't found the right abstraction for 
dealing with software versioning yet.


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Igor Gnatenko]

On Fri, Nov 15, 2019, 17:38 Petr Pisar  wrote:

> On 2019-11-15, Daniel P  Berrangé  wrote:
> > On Fri, Nov 15, 2019 at 02:53:08PM -, Petr Pisar wrote:
> >> On 2019-11-15, John M. Harris Jr  wrote:
> >> > On Friday, November 15, 2019 6:32:21 AM MST Petr Pisar wrote:
> >> >> Example: I have Perl 5.26 as a default version. I have Perl 5.30 as
> an
> >> >> laternative version. Now I want to package Bugzilla that's written in
> >> >> Perl. How do you package Bugzilla so that it works with Perl 5.26 as
> >> >> well as with Perl 5.30?
> >> >
> >> > This sounds like a bug in Modularity.
> >>
> >> Modularity can achieve it when both Perls are packaged as a module. I'm
> >> only showing why we need default stream if we want modules.
> >
> > I'm interested in how this should work when two different modules
> > interact, and we need a language binding across both modules.
> >
> > Consider if we move the virtualization stack (QEMU & Libvirt) into a
> > module with two streams, one libvirt 5.8.0 and one libvirt 6.1.0.
> >
> > Now we want to build Perl bindings for libvirt. We'll need the
> > corresponding version of perl-Sys-Virt either 5.8.0 or 6.1.0,
> > built for each virt module stream, but also built for each Perl
> > module stream 5.26 / 5.30. eg the combinatorial expansion
> >
> >  -   perl-Sys-Virt 5.8.0   with libvirt 5.9.0 with perl 5.26
> >  -   perl-Sys-Virt 5.8.0   with libvirt 5.9.0 with perl 5.30
> >  -   perl-Sys-Virt 6.1.0   with libvirt 6.1.0 with perl 5.26
> >  -   perl-Sys-Virt 6.1.0   with libvirt 6.1.0 with perl 5.30
> >
> True, you have 4 combinations.
>
> > which module would the perl-Sys-Virt builds live in ?
> >
> > If perl-Sys-Virt is part of the virt module, IIUC we'd only be
> > able to build it for one specific perl module stream.
> >
> > If perl-Sys-Virt is part of the perl module, IIUC we'd only be
> > able to build it for one specific virt module stream
> >
> > It looks to me like we have to create a new module just to hold
> > the perl-Sys-Virt package, and give this 4 streams, to cover the
> > combinatorial expansion of the perl & virt module streams. Is
> > this right ?
> >
> No. Modularity solves this combination problem with "stream expansion".
> Sources for such module exists only once, you submit them for building
> with fedpkg only once, but a build systems computes all combinations
> (this the stream expansion) and schedules a build for each of the
> combination. That will result in multiple module builds with the same
> module name, stream, version, but differing with a special
> discriminator called "context".
>

The problem described by Daniel was that Perl module should be different
version when building against 5.x libvirt.

Example: Let's say you have libvirt module with 5.8.0 and 6.1.0 streams
> and perl module with 5.26 and 5.30 streams. If you add perl-Sys-Virt
> into a new module, you write a modulemd file for it like this:
>
> - buildrequiers:
> libvirt: [5.8.0, 6.1.0]
> perl: [5.26, 5.30]
> platform: [f32]
>   requires:
> libvirt: [5.8.0, 6.1.0]
> perl: [5.26, 5.30]
> platform: [f32]
>
> "fedpkg module-build" on it will spawn 4 builds. Even you don't have to
> enumerate the streams and let the module build system to figure out it
> automatically and expand for all existing:
>
> - buildrequiers:
> libvirt: []
> perl: []
> platform: []
>   requires:
> libvirt: []
> perl: []
> platform: []
>
> Or you can put perl-Sys-Virt into libvirt module and write into libvirt
> modulemd of each of the libvirt streams:
>
> - buildrequiers:
> perl: []
> platform: []
>   requires:
> perl: []
> platform: []
>
> You can see these modules in RHEL or CentOS. E.g. perl-DBD-Pg module
> <
> https://git.centos.org/modules/perl-DBD-Pg/blob/c8-stream-3.7/f/perl-DBD-Pg.yaml
> >.
>
> When installing the module, DNF makes sure to select a proper context
> for libvirt and perl you have already selected. If it happened that you
> built it only for Perl 5.26, but you have already enabled Perl 5.24 on
> your system, DNF will report an error that 5.26 module is needed but it
> is disabled.
>
> > And we'd have to do create more modules for every other language
> > binding we ship (ocaml, python, ruby, etc) if the language runtime
> > uses modules.
> >
> You can put all the bindings into one module. Or each binding into its
> own module. Whatever fits your needs better.
>
> -- Petr
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email 

Re: Modularity and the system-upgrade path

[Petr Pisar]

On 2019-11-15, Daniel P  Berrangé  wrote:
> On Fri, Nov 15, 2019 at 02:53:08PM -, Petr Pisar wrote:
>> On 2019-11-15, John M. Harris Jr  wrote:
>> > On Friday, November 15, 2019 6:32:21 AM MST Petr Pisar wrote:
>> >> Example: I have Perl 5.26 as a default version. I have Perl 5.30 as an
>> >> laternative version. Now I want to package Bugzilla that's written in
>> >> Perl. How do you package Bugzilla so that it works with Perl 5.26 as
>> >> well as with Perl 5.30?
>> >
>> > This sounds like a bug in Modularity.
>>
>> Modularity can achieve it when both Perls are packaged as a module. I'm
>> only showing why we need default stream if we want modules.
>
> I'm interested in how this should work when two different modules
> interact, and we need a language binding across both modules.
>
> Consider if we move the virtualization stack (QEMU & Libvirt) into a
> module with two streams, one libvirt 5.8.0 and one libvirt 6.1.0.
>
> Now we want to build Perl bindings for libvirt. We'll need the
> corresponding version of perl-Sys-Virt either 5.8.0 or 6.1.0,
> built for each virt module stream, but also built for each Perl
> module stream 5.26 / 5.30. eg the combinatorial expansion
>
>  -   perl-Sys-Virt 5.8.0   with libvirt 5.9.0 with perl 5.26
>  -   perl-Sys-Virt 5.8.0   with libvirt 5.9.0 with perl 5.30
>  -   perl-Sys-Virt 6.1.0   with libvirt 6.1.0 with perl 5.26
>  -   perl-Sys-Virt 6.1.0   with libvirt 6.1.0 with perl 5.30
>
True, you have 4 combinations.

> which module would the perl-Sys-Virt builds live in ?
>
> If perl-Sys-Virt is part of the virt module, IIUC we'd only be
> able to build it for one specific perl module stream.
>
> If perl-Sys-Virt is part of the perl module, IIUC we'd only be
> able to build it for one specific virt module stream
>
> It looks to me like we have to create a new module just to hold
> the perl-Sys-Virt package, and give this 4 streams, to cover the
> combinatorial expansion of the perl & virt module streams. Is
> this right ?
>
No. Modularity solves this combination problem with "stream expansion".
Sources for such module exists only once, you submit them for building
with fedpkg only once, but a build systems computes all combinations
(this the stream expansion) and schedules a build for each of the
combination. That will result in multiple module builds with the same
module name, stream, version, but differing with a special
discriminator called "context".

Example: Let's say you have libvirt module with 5.8.0 and 6.1.0 streams
and perl module with 5.26 and 5.30 streams. If you add perl-Sys-Virt
into a new module, you write a modulemd file for it like this:

- buildrequiers:
libvirt: [5.8.0, 6.1.0]
perl: [5.26, 5.30]
platform: [f32]
  requires:
libvirt: [5.8.0, 6.1.0]
perl: [5.26, 5.30]
platform: [f32]
   
"fedpkg module-build" on it will spawn 4 builds. Even you don't have to
enumerate the streams and let the module build system to figure out it
automatically and expand for all existing:

- buildrequiers:
libvirt: []
perl: []
platform: []
  requires:
libvirt: []
perl: []
platform: []

Or you can put perl-Sys-Virt into libvirt module and write into libvirt
modulemd of each of the libvirt streams:

- buildrequiers:
perl: []
platform: []
  requires:
perl: []
platform: []

You can see these modules in RHEL or CentOS. E.g. perl-DBD-Pg module
.

When installing the module, DNF makes sure to select a proper context
for libvirt and perl you have already selected. If it happened that you
built it only for Perl 5.26, but you have already enabled Perl 5.24 on
your system, DNF will report an error that 5.26 module is needed but it
is disabled.

> And we'd have to do create more modules for every other language
> binding we ship (ocaml, python, ruby, etc) if the language runtime
> uses modules.
>
You can put all the bindings into one module. Or each binding into its
own module. Whatever fits your needs better.

-- Petr
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[John M. Harris Jr]

On Friday, November 15, 2019 7:53:08 AM MST Petr Pisar wrote:
> Modularity can achieve it when both Perls are packaged as a module. I'm
> only showing why we need default stream if we want modules.

If that's the case, why not build a (separate) Modularity distro? If 
Modularity cannot work with non-modular packages, and that is not a bug with 
Modularity, it is fundamentally incompatible with Fedora as a traditional 
distribution.

> >> If each of the Perls is a stream of a module, you will put Bugzilla into
> >> a module and let it depend on any of the Perls. User can install any of
> >> the Perls and Bugzilla.
> >
> >
> >
> > I'm guessing that Perl from a module doesn't meet a Require on perl?
> 
> 
> It meets the RPM-level "Require on perl". But that's not sufficient
> because every Perl version is not binary compatible. You need to track
> against what Perl Bugzilla was built. That means you need to build Bugzilla
> twice and keep these two Bugzilla builds distinct so that DNF can install
> the right build depending on Perl user has already installed. Modularity
> supports it, but you need both Perl as a module.

That would depend on how the Perl packages are actually handled, which I 
honestly haven't checked, and so I will make no claims as to compatibility.

> >> With your proposal Bugzilla packager would have to package Bugzilla
> >> twice: as a normal package for default Perl 5.26 and as a module for
> >> Perl
> >> 5.30. Then a user would have hard time to select the right combinations
> >> of
> >> Perl and Bugzilla. It would double fork work pacakgers and and make
> >> the system more dificult for users.
> >
> >
> >
> > I don't believe that's the case. The packager would choose how they want
> > to  handle it, most likely just not bothering with modules. The user
> > would just `dnf install bugzilla`, and use the version that is packaged
> > as a non-modular package.
> >
> >
> 
> If packager does not build Bugzilla for the modular Perl, then of course
> the user has no choice. But talk about a case when the user and the
> package wants to have a choice.

It seems, based on what you've said, that Modules remove this choice. If 
somebody chooses for something in the dependency tree to be a module, it all 
has to be a module, otherwise it doesn't work. Please do correct me if I'm 
wrong.

-- 
John M. Harris, Jr.
Splentity

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Miro Hrončok]

On 15. 11. 19 16:11, Petr Pisar wrote:

On 2019-11-15, Miro Hrončok  wrote:

On 15. 11. 19 14:32, Petr Pisar wrote:

On 2019-10-04, Miro Hrončok  wrote:

Wouldn't it be easier if the "default stream" would just behave like
a regular package?

I can think of two solutions of that:

1. (drastic for modular maintainers)

We keep miantaining the default versions of things as ursine packages.
We only modularize alternate versions.


Big con:

That effectively bans modules with multiple dependencies where at least
one is a default version.

Example: I have Perl 5.26 as a default version. I have Perl 5.30 as an
laternative version. Now I want to package Bugzilla that's written in
Perl. How do you package Bugzilla so that it works with Perl 5.26 as
well as with Perl 5.30?


I don't understand why would the user care about the Perl version when
they want Bugzilla. How is Bugzilla different form e.g. Slic3r (app
that happens to be written in Perl)? Do we want to modularize all such
apps to solve the "no parallel instability" feature?


I don't know. Ask the user why he needs a different Perl version than
the default one.  Maybe he has some other applications that work only
with that particular version.


What I was implying is that I don't understand why the user of Buzgilla wants 
different Perl version to run it. I was not implying that users don't want 
various Perl versions generally.



If you believe that users do  not care about a version of software they
use, then we can drop out modularity, and all Fedora releases and
deliver only Rawhide. Or we can stop integrating new versions of
software and deliver Fedora 32 and nothing else forever.


I believe that the purpose of a distribution is to cerate and integrated 
environment, where we simply make sure that Bugzilla works and runs on a Perl 
version we support. And we move forward and integrate with newer Perl versions.


Note that I don't necessarily mean that the use case doesn't exist, I just say I 
don't really get it. And why is Bugzilla any different that all other Perl 
applications.


Either the strategy should be:

"We offer alternate Perl versions for containers etc. they conflict with the 
default Perl version and with the non-modular apps. That is known and accepted."


Or the strategy should be:

"We build all our Perl applications for all our Perl versions, so users who 
choose their Perl stream can still keep their applications from the distribution."


I fail to see what are we trying to achieve here exactly.
It was said several times that parallel instability is a non-goal of Modularity 
and that means certain apps won't install if certain streams are selected. Or 
did I get that wrong?



If each of the Perls is a stream of a module, you will put Bugzilla into
a module and let it depend on any of the Perls. User can install any of
the Perls and Bugzilla.

With your proposal Bugzilla packager would have to package Bugzilla
twice: as a normal package for default Perl 5.26 and as a module for Perl
5.30. Then a user would have hard time to select the right combinations of
Perl and Bugzilla. It would double fork work pacakgers and and make
the system more dificult for users.


With my proposal, Bugzilla packager would package Bugzilla in
non-modular Fedora unless they also want to package it as a module. If
I see correctly, this is exactly the case today.


And do you know the packager does not want to pacakge Bugzilla as
a module? Because in current Fedora without default streams in build
root he had to package it and maintain it twice.


No, I don't. But I know there are packagers of applications who don't want to do 
that. And we should make a distro-scale decision whether we want the whole 
distro to work this way, or whether we only allow this - and those who choose to 
modularize will do so in addition to the non-modular default packages, not instead.


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Daniel P . Berrangé]

On Fri, Nov 15, 2019 at 02:53:08PM -, Petr Pisar wrote:
> On 2019-11-15, John M. Harris Jr  wrote:
> > On Friday, November 15, 2019 6:32:21 AM MST Petr Pisar wrote:
> >> Example: I have Perl 5.26 as a default version. I have Perl 5.30 as an
> >> laternative version. Now I want to package Bugzilla that's written in
> >> Perl. How do you package Bugzilla so that it works with Perl 5.26 as
> >> well as with Perl 5.30?
> >
> > This sounds like a bug in Modularity.
>
> Modularity can achieve it when both Perls are packaged as a module. I'm
> only showing why we need default stream if we want modules.

I'm interested in how this should work when two different modules
interact, and we need a language binding across both modules.

Consider if we move the virtualization stack (QEMU & Libvirt) into a
module with two streams, one libvirt 5.8.0 and one libvirt 6.1.0.

Now we want to build Perl bindings for libvirt. We'll need the
corresponding version of perl-Sys-Virt either 5.8.0 or 6.1.0,
built for each virt module stream, but also built for each Perl
module stream 5.26 / 5.30. eg the combinatorial expansion

 -   perl-Sys-Virt 5.8.0   with libvirt 5.9.0 with perl 5.26
 -   perl-Sys-Virt 5.8.0   with libvirt 5.9.0 with perl 5.30
 -   perl-Sys-Virt 6.1.0   with libvirt 6.1.0 with perl 5.26
 -   perl-Sys-Virt 6.1.0   with libvirt 6.1.0 with perl 5.30

which module would the perl-Sys-Virt builds live in ?

If perl-Sys-Virt is part of the virt module, IIUC we'd only be
able to build it for one specific perl module stream.

If perl-Sys-Virt is part of the perl module, IIUC we'd only be
able to build it for one specific virt module stream

It looks to me like we have to create a new module just to hold
the perl-Sys-Virt package, and give this 4 streams, to cover the
combinatorial expansion of the perl & virt module streams. Is
this right ?

And we'd have to do create more modules for every other language
binding we ship (ocaml, python, ruby, etc) if the language runtime
uses modules.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Petr Pisar]

On 2019-11-15, Miro Hrončok  wrote:
> On 15. 11. 19 14:32, Petr Pisar wrote:
>> On 2019-10-04, Miro Hrončok  wrote:
>>> Wouldn't it be easier if the "default stream" would just behave like
>>> a regular package?
>>>
>>> I can think of two solutions of that:
>>>
>>> 1. (drastic for modular maintainers)
>>>
>>> We keep miantaining the default versions of things as ursine packages.
>>> We only modularize alternate versions.
>>>
>> Big con:
>> 
>> That effectively bans modules with multiple dependencies where at least
>> one is a default version.
>> 
>> Example: I have Perl 5.26 as a default version. I have Perl 5.30 as an
>> laternative version. Now I want to package Bugzilla that's written in
>> Perl. How do you package Bugzilla so that it works with Perl 5.26 as
>> well as with Perl 5.30?
>
> I don't understand why would the user care about the Perl version when
> they want Bugzilla. How is Bugzilla different form e.g. Slic3r (app
> that happens to be written in Perl)? Do we want to modularize all such
> apps to solve the "no parallel instability" feature?
>
I don't know. Ask the user why he needs a different Perl version than
the default one.  Maybe he has some other applications that work only
with that particular version.

If you believe that users do  not care about a version of software they
use, then we can drop out modularity, and all Fedora releases and
deliver only Rawhide. Or we can stop integrating new versions of
software and deliver Fedora 32 and nothing else forever.

>> If each of the Perls is a stream of a module, you will put Bugzilla into
>> a module and let it depend on any of the Perls. User can install any of
>> the Perls and Bugzilla.
>> 
>> With your proposal Bugzilla packager would have to package Bugzilla
>> twice: as a normal package for default Perl 5.26 and as a module for Perl
>> 5.30. Then a user would have hard time to select the right combinations of
>> Perl and Bugzilla. It would double fork work pacakgers and and make
>> the system more dificult for users.
>
> With my proposal, Bugzilla packager would package Bugzilla in
> non-modular Fedora unless they also want to package it as a module. If
> I see correctly, this is exactly the case today.
>
And do you know the packager does not want to pacakge Bugzilla as
a module? Because in current Fedora without default streams in build
root he had to package it and maintain it twice.

-- Petr
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Petr Pisar]

On 2019-11-15, John M. Harris Jr  wrote:
> On Friday, November 15, 2019 6:32:21 AM MST Petr Pisar wrote:
>> Example: I have Perl 5.26 as a default version. I have Perl 5.30 as an
>> laternative version. Now I want to package Bugzilla that's written in
>> Perl. How do you package Bugzilla so that it works with Perl 5.26 as
>> well as with Perl 5.30?
>
> This sounds like a bug in Modularity.
>
Modularity can achieve it when both Perls are packaged as a module. I'm
only showing why we need default stream if we want modules.

>> If each of the Perls is a stream of a module, you will put Bugzilla into
>> a module and let it depend on any of the Perls. User can install any of
>> the Perls and Bugzilla.
>
> I'm guessing that Perl from a module doesn't meet a Require on perl?

It meets the RPM-level "Require on perl". But that's not sufficient
because every Perl version is not binary compatible. You need to track
against what Perl Bugzilla was built. That means you need to build Bugzilla
twice and keep these two Bugzilla builds distinct so that DNF can
install the right build depending on Perl user has already installed.
Modularity supports it, but you need both Perl as a module.

>> With your proposal Bugzilla packager would have to package Bugzilla
>> twice: as a normal package for default Perl 5.26 and as a module for Perl
>> 5.30. Then a user would have hard time to select the right combinations of
>> Perl and Bugzilla. It would double fork work pacakgers and and make
>> the system more dificult for users.
>
> I don't believe that's the case. The packager would choose how they want to 
> handle it, most likely just not bothering with modules. The user would just 
> `dnf install bugzilla`, and use the version that is packaged as a non-modular 
> package.
>
If packager does not build Bugzilla for the modular Perl, then of course
the user has no choice. But talk about a case when the user and the
package wants to have a choice.

-- Petr
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Miro Hrončok]

On 15. 11. 19 14:32, Petr Pisar wrote:

On 2019-10-04, Miro Hrončok  wrote:

Wouldn't it be easier if the "default stream" would just behave like
a regular package?

I can think of two solutions of that:

1. (drastic for modular maintainers)

We keep miantaining the default versions of things as ursine packages.
We only modularize alternate versions.


Big con:

That effectively bans modules with multiple dependencies where at least
one is a default version.

Example: I have Perl 5.26 as a default version. I have Perl 5.30 as an
laternative version. Now I want to package Bugzilla that's written in
Perl. How do you package Bugzilla so that it works with Perl 5.26 as
well as with Perl 5.30?


I don't understand why would the user care about the Perl version when they want 
Bugzilla. How is Bugzilla different form e.g. Slic3r (app that happens to be 
written in Perl)? Do we want to modularize all such apps to solve the "no 
parallel instability" feature?



If each of the Perls is a stream of a module, you will put Bugzilla into
a module and let it depend on any of the Perls. User can install any of
the Perls and Bugzilla.

With your proposal Bugzilla packager would have to package Bugzilla
twice: as a normal package for default Perl 5.26 and as a module for Perl
5.30. Then a user would have hard time to select the right combinations of
Perl and Bugzilla. It would double fork work pacakgers and and make
the system more dificult for users.


With my proposal, Bugzilla packager would package Bugzilla in non-modular Fedora 
unless they also want to package it as a module. If I see correctly, this is 
exactly the case today.


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Petr Pisar]

On 2019-10-07, Jaroslav Mracek  wrote:
> I would like to open discussions more widely, because we are talking about
> future of software distribution and discussing only particular issue is not
> an approach how to delivery solid and stable architecture.
>
> What issues I have in mind?
> 1. Fedora system upgrade (libgit2, axa, bat)
>
I guess this is about changing platform stream without rebuilding
modules. I.e. once relengs create Fedora 32 compose, no modules (except
of virtual "platform") can be installed because they require previous
platform:31.

How it works in classical Fedora? Relengs take Fedora 31 content and
copy it into Fedora 32 repository. And suddenly you have packages
for Fedora 32.

How it could workd in modular Fedora? Relengs take Fedora 31 content
including modules and copy it into Fedora 32 repository. Then they take
modulemd data and rewrite all modular dependencies from platform:31 to
platform:32. Voila, you have modules for Fedora 32. These modulemds
should be imported into MBS to be available at build-time. We don't have
to rebuild the modules because identical packages can belong to more
module builds.

-- Petr
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[John M. Harris Jr]

On Friday, November 15, 2019 6:32:21 AM MST Petr Pisar wrote:
> Example: I have Perl 5.26 as a default version. I have Perl 5.30 as an
> laternative version. Now I want to package Bugzilla that's written in
> Perl. How do you package Bugzilla so that it works with Perl 5.26 as
> well as with Perl 5.30?

This sounds like a bug in Modularity.

> If each of the Perls is a stream of a module, you will put Bugzilla into
> a module and let it depend on any of the Perls. User can install any of
> the Perls and Bugzilla.

I'm guessing that Perl from a module doesn't meet a Require on perl? That's 
not a policy issue, nor an issue with traditional, non-modular, packages.

> With your proposal Bugzilla packager would have to package Bugzilla
> twice: as a normal package for default Perl 5.26 and as a module for Perl
> 5.30. Then a user would have hard time to select the right combinations of
> Perl and Bugzilla. It would double fork work pacakgers and and make
> the system more dificult for users.

I don't believe that's the case. The packager would choose how they want to 
handle it, most likely just not bothering with modules. The user would just 
`dnf install bugzilla`, and use the version that is packaged as a non-modular 
package.

-- 
John M. Harris, Jr.
Splentity

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Petr Pisar]

On 2019-10-04, Miro Hrončok  wrote:
> Wouldn't it be easier if the "default stream" would just behave like
> a regular package?
>
> I can think of two solutions of that:
>
> 1. (drastic for modular maintainers)
>
> We keep miantaining the default versions of things as ursine packages.
> We only modularize alternate versions.
>
Big con:

That effectively bans modules with multiple dependencies where at least
one is a default version.

Example: I have Perl 5.26 as a default version. I have Perl 5.30 as an
laternative version. Now I want to package Bugzilla that's written in
Perl. How do you package Bugzilla so that it works with Perl 5.26 as
well as with Perl 5.30?

If each of the Perls is a stream of a module, you will put Bugzilla into
a module and let it depend on any of the Perls. User can install any of
the Perls and Bugzilla.

With your proposal Bugzilla packager would have to package Bugzilla
twice: as a normal package for default Perl 5.26 and as a module for Perl
5.30. Then a user would have hard time to select the right combinations of
Perl and Bugzilla. It would double fork work pacakgers and and make
the system more dificult for users.

-- Petr
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Zbigniew Jędrzejewski-Szmek]

On Thu, Oct 24, 2019 at 12:42:27PM +0200, Kevin Kofler wrote:
> Neal Gompa wrote:
> 
> > On Thu, Oct 24, 2019 at 4:31 AM Lukas Ruzicka  wrote:
> >> I believe, that if modularity was opt-in, we would be able to use it just
> >> fine, as it is designed now with some little tweakings, such as DNF
> >> providing enough info on retired or discontinued streams, offering a
> >> possibility to choose a different stream to switch to on upgrades. The
> >> longing for default modular Fedora is what makes it more problematic,
> >> because we need to hide everything from the users and make everything
> >> work automatically. If modularity was a matter of personal choice we
> >> would not have to hide anything from anybody, because those users would
> >> be able to read the necessary documentation and tweak their systems just
> >> fine.
> > 
> > Unfortunately there have also been major performance regressions
> > because of the additional work to handle modules being default
> > enabled. The current handling of modules in DNF is not cheap. I'm not
> > sure if this is because it uses libmodulemd1 vs libmodulemd2 or if
> > it's because modules aren't implemented at the libsolv layer and can't
> > be computed as part of the initial constraint set through the base
> > solver. But whatever the reason, it is markedly slower than on systems
> > that don't have modular repositories at all.
> 
> All these are convincing technical arguments for disabling Modularity by 
> default from F32 onwards. (It is unfortunate that these have not been 
> considered for the existing releases, but we cannot turn the time back, we 
> have to focus on improving things for the upcoming releases, hence "from F32 
> onwards". I would even propose it from F31 onwards, but I do not think FESCo 
> is willing to delay the F31 release long enough to implement the required 
> demodularization upgrade path in DNF, hence F32.)

Yes, F32. Doing any significant changes to F31 at this point is out of question.
Please remember that if this path is taken, we'll have to demodularize various
packages, and this will take time too.

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Kevin Kofler]

Neal Gompa wrote:

> On Thu, Oct 24, 2019 at 4:31 AM Lukas Ruzicka  wrote:
>> I believe, that if modularity was opt-in, we would be able to use it just
>> fine, as it is designed now with some little tweakings, such as DNF
>> providing enough info on retired or discontinued streams, offering a
>> possibility to choose a different stream to switch to on upgrades. The
>> longing for default modular Fedora is what makes it more problematic,
>> because we need to hide everything from the users and make everything
>> work automatically. If modularity was a matter of personal choice we
>> would not have to hide anything from anybody, because those users would
>> be able to read the necessary documentation and tweak their systems just
>> fine.
> 
> Unfortunately there have also been major performance regressions
> because of the additional work to handle modules being default
> enabled. The current handling of modules in DNF is not cheap. I'm not
> sure if this is because it uses libmodulemd1 vs libmodulemd2 or if
> it's because modules aren't implemented at the libsolv layer and can't
> be computed as part of the initial constraint set through the base
> solver. But whatever the reason, it is markedly slower than on systems
> that don't have modular repositories at all.

All these are convincing technical arguments for disabling Modularity by 
default from F32 onwards. (It is unfortunate that these have not been 
considered for the existing releases, but we cannot turn the time back, we 
have to focus on improving things for the upcoming releases, hence "from F32 
onwards". I would even propose it from F31 onwards, but I do not think FESCo 
is willing to delay the F31 release long enough to implement the required 
demodularization upgrade path in DNF, hence F32.)

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Neal Gompa]

On Thu, Oct 24, 2019 at 4:31 AM Lukas Ruzicka  wrote:
>
> As far as tooling is concerned, I have been seeing complaints about DNF doing 
> a bad job, but from the perspective of acceptance
> testing, it's the DNF operations that usually work fine with installing, 
> enabling, disabling, removing, resetting and switching modules and streams.
>
> I believe, that if modularity was opt-in, we would be able to use it just 
> fine, as it is designed now with some little tweakings, such as DNF providing 
> enough info on retired or discontinued streams, offering a possibility to 
> choose a different stream to switch to on upgrades. The longing for default 
> modular Fedora is what makes it more problematic, because we need to hide 
> everything from the users and make everything work automatically. If 
> modularity was a matter of personal choice we would not have to hide anything 
> from anybody, because those users would be able to read the necessary 
> documentation and tweak their systems just fine.
>

Unfortunately there have also been major performance regressions
because of the additional work to handle modules being default
enabled. The current handling of modules in DNF is not cheap. I'm not
sure if this is because it uses libmodulemd1 vs libmodulemd2 or if
it's because modules aren't implemented at the libsolv layer and can't
be computed as part of the initial constraint set through the base
solver. But whatever the reason, it is markedly slower than on systems
that don't have modular repositories at all.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Lukas Ruzicka]

This is a policy choice, not a technical matter. If modules became more
> popular, and the dependencies between modules grew, we'd need
> to settle on similar rules, where bigger changes are done with a certain
> cadence. This is why I think that the "independent lifecycles for modules"
> are illusory, made possible by current scarcity of modules.
>

Currently (F31), there are about 63 modules listed with *dnf module list*.
I have attempted to install all modules and all streams. Between single
installations
I always reverted the system to its default state, i.e. modular repos
enabled, but no
modules installed, enabled, disabled or anything.

>From those 63 modules,

   - about 18 are not correctly defined according to criteria that I had
   agreed on with Stephen, https://pagure.io/modularity/issue/149.
   - about 8 modules cannot be installed because of some dependency
   problems: #1764546, #1764616, #1764623, #1764624, #1764606, #1764606,
   #1764611, #1764604

In some of the cases, packagers themselves report that the particular
module should NOT be included
in that particular version of Fedora, currently 31, but they still ARE.

So, not just the tooling, the content is problematic as well, it is not
ready and nobody seems to care, as there
are bugs reported that have not been resolved for several weeks. And since
we do not currently block on modular sanity,
we cannot enforce anything.

As far as tooling is concerned, I have been seeing complaints about DNF
doing a bad job, but from the perspective of acceptance
testing, it's the DNF operations that usually work fine with installing,
enabling, disabling, removing, resetting and switching modules and streams.

I believe, that if modularity was opt-in, we would be able to use it just
fine, as it is designed now with some little tweakings, such as DNF
providing enough info on retired or discontinued streams, offering a
possibility to choose a different stream to switch to on upgrades. The
longing for default modular Fedora is what makes it more problematic,
because we need to hide everything from the users and make everything work
automatically. If modularity was a matter of personal choice we would not
have to hide anything from anybody, because those users would be able to
read the necessary documentation and tweak their systems just fine.

---

Lukáš Růžička

FEDORA QE, RHCE

Red Hat



Purkyňova 115

612 45 Brno - Královo Pole

lruzi...@redhat.com
TRIED AND PERSONALLY TESTED, ERGO TRUSTED. 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Zbigniew Jędrzejewski-Szmek]

On Wed, Oct 23, 2019 at 12:56:41PM -0400, Matthew Miller wrote:
> However, I also have a pretty strong bias towards people who showed up to
> do the work, and the decisions they've made. That doesn't mean we're stuck
> and can't adjust -- in fact, adjusting as we've gone along is a lot of why
> we're where we are now. But unless someone shows up with people-power and
> funding to do it, I take kind of a skeptical view of proposals to start a
> whole new approach from scratch.

Sure, people who show up to do the work get to choose what happens.
But we can't take it to an extreme: there must always be an option to back
out of an idea. Every Change page is required to fill out a Contingency Plan,
and yes, we do occasionally execute those. There are decisions which
need to be implemented for us to see all the benefits and drawbacks, and
in those cases a lot of work is wasted when the contingency plan is enacted.
See for example recent proposal by Ben Cotton to use Taiga: it was 90%
implemented before some drawbacks became visible, and Ben and Manas
took the high road and yanked it.

In fact, the amount of work that has gone into a project is not a
reason to keep trying. If anything, the opposite is true — the more
person-hours have been "consumed" the more that indicates that the
idea is not workable. As we learn from the implementation, we
understand our goals and limitations better, and sometimes we need to
take a hard look and say the expected *remaining* amount of work is
too big. The fact that people showed up and put in work is not the final
consideration.

> > > Because this keeps coming up, we talked about this at the Fedora Council
> > > meeting today. Our goals for modularity are:
> > >   2. Those alternate streams should be able to have different lifecycles.
> >  
> > Hmm, it sounds like the Council hasn't taken into account the constraints
> > on lifecycle of modules that we have slowly discovered during the last
> > two years, constraints that are now part of FESCo-approved policy.
> > 
> > Essentially, modules in Fedora are only allowed to EOL at EOL of Fedora
> > release. And to preserve stability for users, a.k.a. following the Update
> > Policy, modules should only change to new major version at Fedora
> > releases. This is exactly the same as for "normal" rpms.
> 
> This seems appropriate for default steams, but modules should be able to
> have alternate, opt-in streams which either a) update on a rolling or other
> cadence or b) choose to keep building the older version across the release
> boundary.
> 
> The tooling should make this clear to the users.

Yes, default streams, but also streams that other streams or packages depend
on. Having rolling updates or updates with independent cadence is OK if
you are a leaf module and the users opt-in into those changes. But as
soon as people try to build other packages on top, or try to use such modules
in production as dependencies of other things, this breaks down. The
general rule in Fedora is that you get version bumps and non-backwards-compat
behaviour changes between releases, giving users and other packagers a clear
point in time to expect this.
Packages (and streams) can also only be retired at Fedora release EOL. 

This is a policy choice, not a technical matter. If modules became more
popular, and the dependencies between modules grew, we'd need
to settle on similar rules, where bigger changes are done with a certain
cadence. This is why I think that the "independent lifecycles for modules"
are illusory, made possible by current scarcity of modules.

(Or to look at this from another POV: if we want to give users access to
a rolling version of some package, we can do it just as well without modules.
In fact, we already do, with the kernel, with firefox, and probably a bunch
of other packages where this makes sense. For leaf packages this works.
If we want to give users e.g. rolling postgresql, we could provide
postgresql-rolling package. Maybe we should.)

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Matthew Miller]

On Sun, Oct 20, 2019 at 10:47:15AM +, Zbigniew Jędrzejewski-Szmek wrote:
> In general, yes. If the package versions have incompatibilities and/or
> user-visible changes, a different stream is needed for each Fedora
> release. There was a subthread about this recently, starting at

In this case, of course, there needs to be a good way for users to be moved
from stream to stream at upgrade time in a non-disruptive way. I know work
is in progress on that.

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Matthew Miller]

On Sun, Oct 20, 2019 at 09:07:27AM +, Zbigniew Jędrzejewski-Szmek wrote:
> > Because this keeps coming up, we talked about this at the Fedora Council
> > meeting today. Our goals for modularity are:
> >   2. Those alternate streams should be able to have different lifecycles.
>  
> Hmm, it sounds like the Council hasn't taken into account the constraints
> on lifecycle of modules that we have slowly discovered during the last
> two years, constraints that are now part of FESCo-approved policy.
> 
> Essentially, modules in Fedora are only allowed to EOL at EOL of Fedora
> release. And to preserve stability for users, a.k.a. following the Update
> Policy, modules should only change to new major version at Fedora
> releases. This is exactly the same as for "normal" rpms.

This seems appropriate for default steams, but modules should be able to
have alternate, opt-in streams which either a) update on a rolling or other
cadence or b) choose to keep building the older version across the release
boundary.

The tooling should make this clear to the users.

> The lifecycle of modules in Fedora must be the same as lifecycle of
> Fedora releases, so no "different lifecycle" is possible.
> 
> >   1. Users should have alternate streams of software available.
> >   3. Packaging an individual stream for multiple outputs should be easier
> >  than before.
> 
> Those *are* useful goals, but they should not be tied to specific technology,
> we should only care about the end-result.

Yes, that's true from a Council point of view.

However, I also have a pretty strong bias towards people who showed up to
do the work, and the decisions they've made. That doesn't mean we're stuck
and can't adjust -- in fact, adjusting as we've gone along is a lot of why
we're where we are now. But unless someone shows up with people-power and
funding to do it, I take kind of a skeptical view of proposals to start a
whole new approach from scratch.

> Thus, please replace "Our goals for modularity are" with "What we hope
> to achieve with modularity" or even "Our goal is for users to be able to".

I don't really see a meaningful difference there. 

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Stephen John Smoogen]

On Mon, 21 Oct 2019 at 11:08, Randy Barlow  wrote:
>
> On Mon, 2019-10-21 at 09:16 -0400, Stephen John Smoogen wrote:
> > The problem is that COPRs do not have any way of communicating with
> > each other. If I grab from copr-A and it has libfoo-2.3.1-1 and I
> > grab
> > from copr-B and it has libfoo-2.3.2-2 then I am going to replace
> > copr-A's packages which may break what I wanted from there.
>
> Modularity also suffers from this problem.

I don't see it having this problem currently. I can either install one
or the other. I can not mix the two. It may not communicate to me why
I can't mix the two.. but it does stop me from shooting myself in the
foot.




-- 
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Randy Barlow]

On Mon, 2019-10-21 at 14:00 +, Zbigniew Jędrzejewski-Szmek wrote:
> Packaging in Fedora is
> definitely harder than it used to be. We still haven't really
> recovered from
> pkgdb retirement, various infra tools don't have enough support, etc.
> No easy solutions to this problem either, but I think keeping
> packaging
> simple (or at least not more complicated than it currently is).

I agree - I think the retirement of pkgdb without suitable replacement
has made packaging harder when it already wasn't easy. This can't be
helpful towards the goal of growing our contributor base.


signature.asc
Description: This is a digitally signed message part
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Stephen John Smoogen]

On Mon, 21 Oct 2019 at 09:37, Fabio Valentini  wrote:
>
> On Mon, Oct 21, 2019, 15:17 Stephen John Smoogen  wrote:
>>
>> On Mon, 21 Oct 2019 at 01:55, Zbigniew Jędrzejewski-Szmek
>>  wrote:
>> >
>> > On Sun, Oct 20, 2019 at 09:30:52PM -0400, Stephen John Smoogen wrote:
>> > > If I were to start from scratch on this, I would look at the simplest
>> > > solution I would want from Boltron. I want to make it so a package team 
>> > > can
>> > > make a set of packages in a repository and work out how I can interact 
>> > > with
>> > > other repositories. I also want to easily build that package set in ways 
>> > > to
>> > > work on different versions of an operating system.
>> >
>> > The question is whether this team wants to do the "heavy lifting"
>> > (which might or might not actually be very heavy), of integrating with
>> > the rest of the distro. If they don't, then Copr is the answer: it
>> > provides all the answers, including automatic rebuilds.
>> >
>>
>> The problem is that COPRs do not have any way of communicating with
>> each other. If I grab from copr-A and it has libfoo-2.3.1-1 and I grab
>> from copr-B and it has libfoo-2.3.2-2 then I am going to replace
>> copr-A's packages which may break what I wanted from there. I am
>> saying that if we look at a way that they can clearly communicate
>> these problems to the user then we have fixed that.
>
>
> Why not specify those requirements in RPM Requires? That's what they are for.
>

Because this isn't about a package but a set of packages. I may not
know that you are going to use Repo-B.. but may have known about
Repo-C which did work. [Maybe it turns out that libfoo-2.3.2-2 in repo
C does work because it had a patch.]

The problem I am seeing is that with N thousand copr packages, you can
only express some amount of stuff in the packages themselves. In the
end there are parts which need to be expressed more abstractly higher
up. [I know I work with copr-X, I don't work with copr-Y.] etc.

This is a common problem which shows up because people have problems
they want to solve, they google to find a solution and htey cobble
together something from 10 different repositories which may or may not
have been designed to work together. [This will also happen in
container combinations etc so if we can help fix it here.. it can be
used elsewhere.]

The answers of 'well dont' do that', 'get them to put them in the OS',
etc etc have not worked for 20 years.

The idea of adding more packagers to the community hasn't worked
either.. even before pkgdb was retired.. there was no 'growth'.. there
was a set number of people who were active (they may not be the same
as now.. but the number was the same). I would say 350 active
packagers is our Dunbar number but that is getting far into the
pseudo-science weeds. Is that the number of packagers? No.. there are
a lot of people who will do a small set and have no interested outside
of that. Giving them the tools which can allow them to communicate
what their repo can and can not do seems like a good thing.

-- 
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Randy Barlow]

On Mon, 2019-10-21 at 14:00 +, Zbigniew Jędrzejewski-Szmek wrote:
> The problem is hard. If there was an obvious solution, we wouldn't be
> having this discussion.

I've pointed out a few times that other distros have solved the "too
fast, too slow" problem. In at least one case, as long ago as 2004. I
see it as a solved problem and I don't understand why we are trying to
solve it again.


signature.asc
Description: This is a digitally signed message part
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Randy Barlow]

On Mon, 2019-10-21 at 09:16 -0400, Stephen John Smoogen wrote:
> The problem is that COPRs do not have any way of communicating with
> each other. If I grab from copr-A and it has libfoo-2.3.1-1 and I
> grab
> from copr-B and it has libfoo-2.3.2-2 then I am going to replace
> copr-A's packages which may break what I wanted from there.

Modularity also suffers from this problem.


signature.asc
Description: This is a digitally signed message part
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Orion Poplawski]

On 10/21/19 7:16 AM, Stephen John Smoogen wrote:

On Mon, 21 Oct 2019 at 01:55, Zbigniew Jędrzejewski-Szmek
 wrote:


On Sun, Oct 20, 2019 at 09:30:52PM -0400, Stephen John Smoogen wrote:

If I were to start from scratch on this, I would look at the simplest
solution I would want from Boltron. I want to make it so a package team can
make a set of packages in a repository and work out how I can interact with
other repositories. I also want to easily build that package set in ways to
work on different versions of an operating system.


The question is whether this team wants to do the "heavy lifting"
(which might or might not actually be very heavy), of integrating with
the rest of the distro. If they don't, then Copr is the answer: it
provides all the answers, including automatic rebuilds.



The problem is that COPRs do not have any way of communicating with
each other. If I grab from copr-A and it has libfoo-2.3.1-1 and I grab
from copr-B and it has libfoo-2.3.2-2 then I am going to replace
copr-A's packages which may break what I wanted from there. I am
saying that if we look at a way that they can clearly communicate
these problems to the user then we have fixed that.



FWIW, there is this old bug asking to express at least some kind of 
dependency between COPRs:

 https://bugzilla.redhat.com/show_bug.cgi?id=1149887

--
Orion Poplawski
Manager of NWRA Technical Systems  720-772-5637
NWRA, Boulder/CoRA Office FAX: 303-415-9702
3380 Mitchell Lane   or...@nwra.com
Boulder, CO 80301 https://www.nwra.com/



smime.p7s
Description: S/MIME Cryptographic Signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Zbigniew Jędrzejewski-Szmek]

On Mon, Oct 21, 2019 at 03:36:53PM +0200, Fabio Valentini wrote:
> On Mon, Oct 21, 2019, 15:17 Stephen John Smoogen  wrote:
> 
> > On Mon, 21 Oct 2019 at 01:55, Zbigniew Jędrzejewski-Szmek
> >  wrote:
> > >
> > > On Sun, Oct 20, 2019 at 09:30:52PM -0400, Stephen John Smoogen wrote:
> > > > If I were to start from scratch on this, I would look at the simplest
> > > > solution I would want from Boltron. I want to make it so a package
> > team can
> > > > make a set of packages in a repository and work out how I can interact
> > with
> > > > other repositories. I also want to easily build that package set in
> > ways to
> > > > work on different versions of an operating system.
> > >
> > > The question is whether this team wants to do the "heavy lifting"
> > > (which might or might not actually be very heavy), of integrating with
> > > the rest of the distro. If they don't, then Copr is the answer: it
> > > provides all the answers, including automatic rebuilds.
> > >
> >
> > The problem is that COPRs do not have any way of communicating with
> > each other. If I grab from copr-A and it has libfoo-2.3.1-1 and I grab
> > from copr-B and it has libfoo-2.3.2-2 then I am going to replace
> > copr-A's packages which may break what I wanted from there. I am
> > saying that if we look at a way that they can clearly communicate
> > these problems to the user then we have fixed that.
> >
> 
> Why not specify those requirements in RPM Requires? That's what they are
> for.

Exactly.

(Though, I don't think it is wise to build something complicated from
coprs — they are intended to allow the rules to be relaxed, but the obvious
corollary is that there is less stability and reliability. The rules we
have in the distro how packages must behave might be stiffling and annoying,
but they are there for a reason...)

> > Also there needs to be a way to communicate that an upgrade from F32
> > to F33 will break a system because copr-B has no F-33 packages.
> >
> 
> This already works somewhat, the only change that would be needed is
> setting skip_if_unavailable = false for COPR repos (I think they're set to
> true right now).
> 
> 
> > > If they do, and they invest in following the packaging guidelines and
> > > and the release cycles and whatever we say makes the package suitable
> > > for users and other packagers to build on, they get to put the package
> > > in the distro.
> > >
> >
> > From what I have heard over and over is that it isn't the packaging
> > guidelines which are a problem.. it is dealing with threads like this
> > or the continual drama churn we have. Investing in the OS means a lot
> > of emotional energy which a lot of people have no room for in our
> > current world. In some ways I see being able to bolt things into Coprs
> > as an escape from dealing with constant absolutes of 'your wrong!'
> > which most of our messages devolve to.

I consider our discussions to be technical and at a good level. Even
this thread: there have a been a few flare-ups, but that's just a handful,
and just people being tired and putting a few words in the wrong place
rather than any personal attack.

The problem is hard. If there was an obvious solution, we wouldn't be
having this discussion.

> >
> > The problem is that our current 20,000 packages is a LOT and most
> > software needs more than we actually have packaged. That means
> > continual growth, but our other needs of 'I need this as quickly as
> > possible', 'I expect you to have fixed all these things', etc are more
> > than most volunteers can deal with at this size. We end up shutting
> > down and yelling at each other because deep down we just want the
> > noise to stop.
> >
> 
> Yes, I agree, the current growth of the package set isn't sustainable if we
> don't also scale up the contributor base. I suspect that there are a few
> handfuls of packagers who maintain hundreds of packages, while the majority
> only maintains only a handful of packages. And relying on the
> "overcommitters" (pun intended) to keep the distro running isn't working so
> well.

I very much hope we can grow our packager base. Packaging in Fedora is
definitely harder than it used to be. We still haven't really recovered from
pkgdb retirement, various infra tools don't have enough support, etc.
No easy solutions to this problem either, but I think keeping packaging
simple (or at least not more complicated than it currently is).

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Fabio Valentini]

On Mon, Oct 21, 2019, 15:17 Stephen John Smoogen  wrote:

> On Mon, 21 Oct 2019 at 01:55, Zbigniew Jędrzejewski-Szmek
>  wrote:
> >
> > On Sun, Oct 20, 2019 at 09:30:52PM -0400, Stephen John Smoogen wrote:
> > > If I were to start from scratch on this, I would look at the simplest
> > > solution I would want from Boltron. I want to make it so a package
> team can
> > > make a set of packages in a repository and work out how I can interact
> with
> > > other repositories. I also want to easily build that package set in
> ways to
> > > work on different versions of an operating system.
> >
> > The question is whether this team wants to do the "heavy lifting"
> > (which might or might not actually be very heavy), of integrating with
> > the rest of the distro. If they don't, then Copr is the answer: it
> > provides all the answers, including automatic rebuilds.
> >
>
> The problem is that COPRs do not have any way of communicating with
> each other. If I grab from copr-A and it has libfoo-2.3.1-1 and I grab
> from copr-B and it has libfoo-2.3.2-2 then I am going to replace
> copr-A's packages which may break what I wanted from there. I am
> saying that if we look at a way that they can clearly communicate
> these problems to the user then we have fixed that.
>

Why not specify those requirements in RPM Requires? That's what they are
for.


> Also there needs to be a way to communicate that an upgrade from F32
> to F33 will break a system because copr-B has no F-33 packages.
>

This already works somewhat, the only change that would be needed is
setting skip_if_unavailable = false for COPR repos (I think they're set to
true right now).


> > If they do, and they invest in following the packaging guidelines and
> > and the release cycles and whatever we say makes the package suitable
> > for users and other packagers to build on, they get to put the package
> > in the distro.
> >
>
> From what I have heard over and over is that it isn't the packaging
> guidelines which are a problem.. it is dealing with threads like this
> or the continual drama churn we have. Investing in the OS means a lot
> of emotional energy which a lot of people have no room for in our
> current world. In some ways I see being able to bolt things into Coprs
> as an escape from dealing with constant absolutes of 'your wrong!'
> which most of our messages devolve to.
>
> The problem is that our current 20,000 packages is a LOT and most
> software needs more than we actually have packaged. That means
> continual growth, but our other needs of 'I need this as quickly as
> possible', 'I expect you to have fixed all these things', etc are more
> than most volunteers can deal with at this size. We end up shutting
> down and yelling at each other because deep down we just want the
> noise to stop.
>

Yes, I agree, the current growth of the package set isn't sustainable if we
don't also scale up the contributor base. I suspect that there are a few
handfuls of packagers who maintain hundreds of packages, while the majority
only maintains only a handful of packages. And relying on the
"overcommitters" (pun intended) to keep the distro running isn't working so
well.

Fabio


>
>
>
> --
> Stephen J Smoogen.
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Stephen John Smoogen]

On Mon, 21 Oct 2019 at 01:55, Zbigniew Jędrzejewski-Szmek
 wrote:
>
> On Sun, Oct 20, 2019 at 09:30:52PM -0400, Stephen John Smoogen wrote:
> > If I were to start from scratch on this, I would look at the simplest
> > solution I would want from Boltron. I want to make it so a package team can
> > make a set of packages in a repository and work out how I can interact with
> > other repositories. I also want to easily build that package set in ways to
> > work on different versions of an operating system.
>
> The question is whether this team wants to do the "heavy lifting"
> (which might or might not actually be very heavy), of integrating with
> the rest of the distro. If they don't, then Copr is the answer: it
> provides all the answers, including automatic rebuilds.
>

The problem is that COPRs do not have any way of communicating with
each other. If I grab from copr-A and it has libfoo-2.3.1-1 and I grab
from copr-B and it has libfoo-2.3.2-2 then I am going to replace
copr-A's packages which may break what I wanted from there. I am
saying that if we look at a way that they can clearly communicate
these problems to the user then we have fixed that.

Also there needs to be a way to communicate that an upgrade from F32
to F33 will break a system because copr-B has no F-33 packages.

> If they do, and they invest in following the packaging guidelines and
> and the release cycles and whatever we say makes the package suitable
> for users and other packagers to build on, they get to put the package
> in the distro.
>

From what I have heard over and over is that it isn't the packaging
guidelines which are a problem.. it is dealing with threads like this
or the continual drama churn we have. Investing in the OS means a lot
of emotional energy which a lot of people have no room for in our
current world. In some ways I see being able to bolt things into Coprs
as an escape from dealing with constant absolutes of 'your wrong!'
which most of our messages devolve to.

The problem is that our current 20,000 packages is a LOT and most
software needs more than we actually have packaged. That means
continual growth, but our other needs of 'I need this as quickly as
possible', 'I expect you to have fixed all these things', etc are more
than most volunteers can deal with at this size. We end up shutting
down and yelling at each other because deep down we just want the
noise to stop.




-- 
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Lukas Ruzicka]

>
>
>
> This has been working quite fine so far, so it would be bad to loose this
> capability, as the actual
> users in question are definitely not power users and will not be able to
> fix any of these issues
> by themselves.
>
>
+1, this was one of my points, too. I think that Fedora should be open for
users of all categories, not just
power users, but also newbies and similar species.



-- 

Lukáš Růžička

FEDORA QE, RHCE

Red Hat



Purkyňova 115

612 45 Brno - Královo Pole

lruzi...@redhat.com
TRIED AND PERSONALLY TESTED, ERGO TRUSTED. 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Martin Kolman]

On Fri, 2019-10-18 at 13:05 +0200, Lukas Ruzicka wrote:
> 
> > Or, even better (or worse): Sombody installs GIMP via GNOME Software,
> > 
> > and under the hood, dnf does its magic and installs gimp from the
> > 
> > module, which might depend on another, even non-default module, etc.
> > 
> > But then, what will happen when that module is EOL, and the user has
> > 
> > never even interacted with dnf, or modules? Will system upgrades break
> > 
> > and prompt the user to fix something they didn't even know existed,
> > 
> > and have never interacted with?
> 
> This has already happened. We have had complaints from people who had never
> installed any module, or used the "dnf module" command and they still ended 
> up with
> modules installed with a problem to upgrade their computers to F31.
This worries me quite a bit as people often install Fedora on computers of 
relatives and friendsin default configuration
& instruct the users to just "press the update button when it shows up".I've 
done that a couple times.
This has been working quite fine so far, so it would be bad to loose this 
capability, as the actualusers in question are
definitely not power users and will not be able to fix any of these issuesby 
themselves. 
Also from their point of view it's pretty much Fedora breaking if if a specific 
module is ta fault.And well, they are
not really wrong if a default Fedora install simply breaks upgrades at arandom 
point in the future...
>  
> > 
> > Fabio
> > 
> > 
> > 
> > > Pierre
> > 
> > > ___
> > 
> > > devel mailing list -- devel@lists.fedoraproject.org
> > 
> > > To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> > 
> > > Fedora Code of Conduct: 
> > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > 
> > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > 
> > > List Archives: 
> > > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> > 
> > ___
> > 
> > devel mailing list -- devel@lists.fedoraproject.org
> > 
> > To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> > 
> > Fedora Code of Conduct: 
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > 
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > 
> > List Archives: 
> > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> > 
> 
> 
> ___devel mailing list -- 
> devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[John M. Harris Jr]

On Thursday, October 17, 2019 4:28:27 PM MST Kevin Kofler wrote:
> Adam Williamson wrote:
> 
> > Of course if you just don't modularize FreeIPA at all you don't have
> > the kickstart problem, but then you *do* still have the 'we're stuck
> > shipping this one version of FreeIPA for the next seventy jillion
> > years' problem.
> 
> 
> That is purely a RHEL thing though. I do not see how this is relevant to the
>  discussion on whether to allow default streams *in Fedora*.

Even then, as a RHEL subscriber, I'd question its usefulness there. I use RHEL 
at work because I want a solid stable system without many changes in a release 
cycle. It makes is much easier with large deployments and high numbers of 
administrators/users, who may be resistant to change. Throwing a random new 
version of a package on there sounds like a nightmare.

Actually, I'm not even sure how I'd deploy modules in the environment that I 
support, because it doesn't have internet access. I usually just download the 
RHEL repo DVD and I'm good to go.

-- 
John M. Harris, Jr.
Splentity

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Vít Ondruch]

Dne 18. 10. 19 v 17:04 Matthew Miller napsal(a):
> On Fri, Oct 18, 2019 at 01:03:24PM +0200, Lukas Ruzicka wrote:
>> Exactly ... this is what I believe, too. I think that Fedora users put
>> Fedora on their desktops and laptops to be creative in many ways of
>> creativity. Some make make music, some enhance pictures, some model in
>> Blender, cut videos, write documents. The majority, I dare to say, is not
>> interested in having several Inkscape versions, they want the newest yet
>> stable enough and they are satisfied with that.
> Well, maybe. Here's an actual Fedora story. A few releases ago, we had a
> designer working on a little animation promoting how trouble-free and
> painless Fedora updates are now (as a response to the "you should do an LTS,
> or a rolling release!" messages I often here).
>
> But, in the middle of making this, they updated themselves, and the new
> version of Inkscape dropped support for a feature (a file format, I think)
> that was very important to their work —  so they said that they couldn't
> continue making that ad in good conscience.
>
> If we had Inkscape as two streams, independent of the OS release, they could
> have opted to continue with the one that worked for them for a while longer
> at least.
>
>

... or the stream they used would go EOL at the same time they upgraded
and they would have no option. Even if it was not EOL, they would need
to fiddle with choosing the right module which is BTW not possible in
G-S if I am not mistaken.

I think we should be more careful about promoting modules and they
should be carefully considered.


Vít

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Zbigniew Jędrzejewski-Szmek]

On Sun, Oct 20, 2019 at 09:30:52PM -0400, Stephen John Smoogen wrote:
> If I were to start from scratch on this, I would look at the simplest
> solution I would want from Boltron. I want to make it so a package team can
> make a set of packages in a repository and work out how I can interact with
> other repositories. I also want to easily build that package set in ways to
> work on different versions of an operating system.

The question is whether this team wants to do the "heavy lifting"
(which might or might not actually be very heavy), of integrating with
the rest of the distro. If they don't, then Copr is the answer: it
provides all the answers, including automatic rebuilds.

If they do, and they invest in following the packaging guidelines and
and the release cycles and whatever we say makes the package suitable
for users and other packagers to build on, they get to put the package
in the distro.

> 1. They have a lot of packages they need to tie together
> 2. They need to build those packages for multiple deliverable operating
> environments
> 3. They need to interact with other groups of packages in a way which that
> their group can 'know' if there is a chance of coworking.
> 4. Many are tied to systems which have fast, hard update cycles which do
> not work with even a 'fast' OS like Fedora.
> 
> Users are wanting
> 1. A system which can tie these different speed things together.
> 2. That system to be updated or is clear on why it can't and what needs to
> be done.

This is all already satisfied by rpms (even from Copr).
In particular, for point 3., there can be no magic: we can *express*
relationships between packages with Provides and Requires, but to
*know* what should be expressed, we need packager input _and_ ideally
lots of QA and testing. No new repo format and metadata language fundamentally
changes this.

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Stephen John Smoogen]

On Sun, Oct 20, 2019 at 15:20 Zbigniew Jędrzejewski-Szmek 
wrote:

> On Tue, Oct 15, 2019 at 09:26:31PM -0400, Stephen Gallagher wrote:
> > Alternate Proposal:
> >
> > Most things from the original proposal in the first message of this
> > thread remains the same except:
> >
> > Module stream metadata would gain two new optional attributes,
> > "upgrades:" and "obsoletes:".
>
> I'm sorry, but I'm against this proposal, both in its first form, and
> as amended here. The long discussion in this thread has pushed me over
> into conviction that modules should not be "on by default" in any way
> in Fedora.
>
> The first form of the proposal was already staggeringly complex —
> "default", "dep_enabled", "default_enabled", "default", …. Recording
> user intent when the users interacts directly with the thing
> might be OK, but mapping that intent onto dependencies that are pulled
> in automatically is not something that can be well defined. My expectation
> is that we'd forever be fighting broken expectations and unexpected cases.
>
> But the amended proposal actually makes things *worse*, even more complex.
> We would have two parallel sets of dependency specifications: on the
> rpms level and on the module level. The interactions between them would
> be hard to understand for users.
>
> I also don't think we need this at all: everything that could be
> expressed using deps between modules can also be expressed using deps
> between rpms. We have a rich and well defined dependency language for
> rpms, so let's just use it.
>
> One of the operational problems with Modularity is that it places huge
> expectations on dnf. Modularity was never very well defined, and dnf
> had to adapt on the fly to changing rules and requirements. This never
> went well. Even more complexity, with three parallel sets of
> semi-interacting-semi-independent sets of constraint rules (rpm deps,
> module intent, module obsoletes+provides), expressed in different form,
> is imho a recipe for bad ux.
>
> At the same time, this thread has shown that this additional
> complexity would need to be added to have upgrade paths for modules.
> Essentially, to me this thread has shown that Modularity needs to go
> back to drawing board, to reassess goals and assumptions and
> implementation choices. A lot of what people *thought* Modularity
> would give them, is simply not possible, and at the same time, the
> costs and impact on the rest of the distribution is higher than
> expected.
>
> As has been extensively discussed, modules are opaque and a) by design
> make some rpms not visible and not as available to other packagers as
> before, b) make it harder for people outside of Fedora to reuse our
> packaging and build on top of Fedora.
>
> Modules also raise the complexity of packaging. I understand that for
> some expert packagers they provide new functionality, but they complicate
> life for the majority of packagers. I think this additional complexity
> is of the reasons for the decline in participation of non-expert
> packagers in Fedora that was show in FPL's graphs.
>
> The work that went into Modularity is certainly not all wasted: I think
> we all understand the problem and what solutions don't work much better
> then before. I think we should take a step back and try for a solution
> which has lower end-user complexity and better backwards compatibility.
>
> I'm not asking for an improved proposal here: for me, Modularity in its
> current form is simply not a net benefit for Fedora's packagers or users.
> Thus, I'm against both default modules, and adding modules in the
> buildroot, and against rebasing any part of Fedora to build on top of
> modules. This is "contingency mode", i.e. thinking how to bring things
> back to working state. I think it is OK to allow modules to be available,
> but they must be opt-in, and normal rpms may not allow on the
> modularized rpms in any way.
>
> I wrote this in reply to this thread, even though some things might
> fit more in the sister thread "Fedora 32 System-Wide Change proposal:
> Modules in Non-Modular Buildroot". I don't want to send two mails with
> a lot of text, and the two things are inextricably linked: we cannot
> enable modules by default, or make more things depend on them by
> including in the build root, without having working upgrade paths.
>

If I were to start from scratch on this, I would look at the simplest
solution I would want from Boltron. I want to make it so a package team can
make a set of packages in a repository and work out how I can interact with
other repositories. I also want to easily build that package set in ways to
work on different versions of an operating system.  Let us ignore the early
goal of modularity of ‘I don’t want a ton of repositories’ which led to
‘virtual repositories’ which led to ‘modules’. For our ‘persona’ story..
let us go back to the days of Dag and Athimm .. and we want to be able to
give them the tools to build 2 repositories which worked on 

Re: Modularity and the system-upgrade path

[Zbigniew Jędrzejewski-Szmek]

On Tue, Oct 15, 2019 at 09:26:31PM -0400, Stephen Gallagher wrote:
> Alternate Proposal:
> 
> Most things from the original proposal in the first message of this
> thread remains the same except:
> 
> Module stream metadata would gain two new optional attributes,
> "upgrades:" and "obsoletes:".

I'm sorry, but I'm against this proposal, both in its first form, and
as amended here. The long discussion in this thread has pushed me over
into conviction that modules should not be "on by default" in any way
in Fedora.

The first form of the proposal was already staggeringly complex —
"default", "dep_enabled", "default_enabled", "default", …. Recording
user intent when the users interacts directly with the thing
might be OK, but mapping that intent onto dependencies that are pulled
in automatically is not something that can be well defined. My expectation
is that we'd forever be fighting broken expectations and unexpected cases.

But the amended proposal actually makes things *worse*, even more complex.
We would have two parallel sets of dependency specifications: on the
rpms level and on the module level. The interactions between them would
be hard to understand for users.

I also don't think we need this at all: everything that could be
expressed using deps between modules can also be expressed using deps
between rpms. We have a rich and well defined dependency language for
rpms, so let's just use it.

One of the operational problems with Modularity is that it places huge
expectations on dnf. Modularity was never very well defined, and dnf
had to adapt on the fly to changing rules and requirements. This never
went well. Even more complexity, with three parallel sets of
semi-interacting-semi-independent sets of constraint rules (rpm deps,
module intent, module obsoletes+provides), expressed in different form,
is imho a recipe for bad ux.

At the same time, this thread has shown that this additional
complexity would need to be added to have upgrade paths for modules.
Essentially, to me this thread has shown that Modularity needs to go
back to drawing board, to reassess goals and assumptions and
implementation choices. A lot of what people *thought* Modularity
would give them, is simply not possible, and at the same time, the
costs and impact on the rest of the distribution is higher than
expected.

As has been extensively discussed, modules are opaque and a) by design
make some rpms not visible and not as available to other packagers as
before, b) make it harder for people outside of Fedora to reuse our
packaging and build on top of Fedora.

Modules also raise the complexity of packaging. I understand that for
some expert packagers they provide new functionality, but they complicate
life for the majority of packagers. I think this additional complexity
is of the reasons for the decline in participation of non-expert
packagers in Fedora that was show in FPL's graphs.

The work that went into Modularity is certainly not all wasted: I think
we all understand the problem and what solutions don't work much better
then before. I think we should take a step back and try for a solution
which has lower end-user complexity and better backwards compatibility.

I'm not asking for an improved proposal here: for me, Modularity in its
current form is simply not a net benefit for Fedora's packagers or users.
Thus, I'm against both default modules, and adding modules in the
buildroot, and against rebasing any part of Fedora to build on top of
modules. This is "contingency mode", i.e. thinking how to bring things
back to working state. I think it is OK to allow modules to be available,
but they must be opt-in, and normal rpms may not allow on the
modularized rpms in any way.

I wrote this in reply to this thread, even though some things might
fit more in the sister thread "Fedora 32 System-Wide Change proposal:
Modules in Non-Modular Buildroot". I don't want to send two mails with
a lot of text, and the two things are inextricably linked: we cannot
enable modules by default, or make more things depend on them by
including in the build root, without having working upgrade paths.

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Petr Stodulka]

On 15. 10. 19 19:25, Miro Hrončok wrote:

On 15. 10. 19 19:13, Kevin Fenzi wrote:

So, I see the following modules (in rawhide anyhow) that don't seem to
have non modular versions:

avocado
cri-o
django
dwm
eclipse
gimp
jmc
lizardfs
mysql
ninja
perl-bootstrap
stratis


Do all of those have default streams? I don't know all but I think that at least gimp, 
django, mysql have nonmodular "dafault" versions.

In case of gimp, it has both nonmodular version and a default modular stream.

perl-bootstrap is probably needed only to bootstrap Perl.

Eclipse has a nonmodular version that fails to install because other stuff has 
default streams.

Yes, some of the modules would need to be "converted back". Better to do it 
when we can still count them on our fingers.


+1

--
Petr Stodulka
Core Services (In-place upgrades and migrations)
Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Zbigniew Jędrzejewski-Szmek]

On Sun, Oct 20, 2019 at 10:47:15AM +, Zbigniew Jędrzejewski-Szmek wrote:
> On Sun, Oct 20, 2019 at 11:35:37AM +0200, Fabio Valentini wrote:
> > On Sun, Oct 20, 2019 at 11:09 AM Zbigniew Jędrzejewski-Szmek
> >  wrote:
> > >
> > > On Wed, Oct 16, 2019 at 02:48:13PM -0400, Matthew Miller wrote:
> > > > On Wed, Oct 16, 2019 at 08:31:10AM -0400, Stephen Gallagher wrote:
> > > > > This is not true. It should be *possible* to have a fully modularized
> > > > > distribution, but that isn't a specific goal for Fedora or RHEL.
> > > >
> > > > Because this keeps coming up, we talked about this at the Fedora Council
> > > > meeting today. Our goals for modularity are:
> > > >   2. Those alternate streams should be able to have different 
> > > > lifecycles.
> > >
> > > Hmm, it sounds like the Council hasn't taken into account the constraints
> > > on lifecycle of modules that we have slowly discovered during the last
> > > two years, constraints that are now part of FESCo-approved policy.
> > >
> > > Essentially, modules in Fedora are only allowed to EOL at EOL of Fedora 
> > > release.
> > > And to preserve stability for users, a.k.a. following the Update Policy,
> > > modules should only change to new major version at Fedora releases.
> > > This is exactly the same as for "normal" rpms.
> > >
> > > The lifecycle of modules in Fedora must be the same as lifecycle of
> > > Fedora releases, so no "different lifecycle" is possible.
> > 
> > Ok, just to be sure that I understand this correctly:
> > 
> > - module EOL dates must align with fedora release EOL dates,
> 
> Yes, this was voted in https://pagure.io/modularity/issue/112#comment-553234
> > Allow maintainers to specify that a module stream will live until
> > the EOL date of a particular Fedora release or EPEL minor release,
> > with special cases for "just keep building until I say otherwise"?
> and approved in https://pagure.io/modularity/issue/112#comment-562677.
> (I'm providing exact links because it's hard to find.)
> 
> > - Update Policy is the same for modules as for normal packages,
> > - major package updates can only occur at "release upgrade" time
> 
> I'm not sure if that is specified in plain text anywhere.
> The last image in 
> https://pagure.io/modularity/working-documents/blob/master/f/lifecycles-upgrades-ownership/lifecycles-general.md
> shows that at least.
> 
> But the gist of 
> https://docs.fedoraproject.org/en-US/fesco/Updates_Policy/#philosophy
> applies to modules too: if there's a module that has been released for
> some Fedora version, a major user-visible change would be just a disruptive
> for users as a major user-visible change in any packages. This certainly
> applies to streams like "/stable" and "/version-nnn".
> 
> Maybe somebody from the Modularity team can provide clarification here
> and links to policy.
> 
> > If I'm not suffering from too low blood levels of caffeine right now,
> > then from these 3 constraints follows:
> > 
> > - default streams are basically useless (since they cannot target
> > multiple fedora releases in most cases, due to the Update Policy),
> 
> In general, yes. If the package versions have incompatibilities and/or
> user-visible changes, a different stream is needed for each Fedora
> release. There was a subthread about this recently, starting at
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/C5EX4WI7Z4HZZLVTUIHSMMPPUGTFENYE/.

Wait, did I just reply to you quoting your earlier mail? Oops,
I did. Smooge is right, this thread has started looping.

Zbyszek

> > - flexible lifecycle advantages of modules do not apply to fedora,
> > since module EOL dates must align with fedora release EOL dates.
> > 
> > Then, what *is* the benefit of using modules for "default" versions of
> > fedora packages, if "default" streams have to usually be maintained
> > separately for different fedora branches, just like normal packages,
> > but with the *additional* overhead of Modularity - and additional work
> > for maintainers of dependent packages?
> 
> That is one of questions we are trying to answer in this thread ;)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Zbigniew Jędrzejewski-Szmek]

On Sun, Oct 20, 2019 at 11:35:37AM +0200, Fabio Valentini wrote:
> On Sun, Oct 20, 2019 at 11:09 AM Zbigniew Jędrzejewski-Szmek
>  wrote:
> >
> > On Wed, Oct 16, 2019 at 02:48:13PM -0400, Matthew Miller wrote:
> > > On Wed, Oct 16, 2019 at 08:31:10AM -0400, Stephen Gallagher wrote:
> > > > This is not true. It should be *possible* to have a fully modularized
> > > > distribution, but that isn't a specific goal for Fedora or RHEL.
> > >
> > > Because this keeps coming up, we talked about this at the Fedora Council
> > > meeting today. Our goals for modularity are:
> > >   2. Those alternate streams should be able to have different lifecycles.
> >
> > Hmm, it sounds like the Council hasn't taken into account the constraints
> > on lifecycle of modules that we have slowly discovered during the last
> > two years, constraints that are now part of FESCo-approved policy.
> >
> > Essentially, modules in Fedora are only allowed to EOL at EOL of Fedora 
> > release.
> > And to preserve stability for users, a.k.a. following the Update Policy,
> > modules should only change to new major version at Fedora releases.
> > This is exactly the same as for "normal" rpms.
> >
> > The lifecycle of modules in Fedora must be the same as lifecycle of
> > Fedora releases, so no "different lifecycle" is possible.
> 
> Ok, just to be sure that I understand this correctly:
> 
> - module EOL dates must align with fedora release EOL dates,

Yes, this was voted in https://pagure.io/modularity/issue/112#comment-553234
> Allow maintainers to specify that a module stream will live until
> the EOL date of a particular Fedora release or EPEL minor release,
> with special cases for "just keep building until I say otherwise"?
and approved in https://pagure.io/modularity/issue/112#comment-562677.
(I'm providing exact links because it's hard to find.)

> - Update Policy is the same for modules as for normal packages,
> - major package updates can only occur at "release upgrade" time

I'm not sure if that is specified in plain text anywhere.
The last image in 
https://pagure.io/modularity/working-documents/blob/master/f/lifecycles-upgrades-ownership/lifecycles-general.md
shows that at least.

But the gist of 
https://docs.fedoraproject.org/en-US/fesco/Updates_Policy/#philosophy
applies to modules too: if there's a module that has been released for
some Fedora version, a major user-visible change would be just a disruptive
for users as a major user-visible change in any packages. This certainly
applies to streams like "/stable" and "/version-nnn".

Maybe somebody from the Modularity team can provide clarification here
and links to policy.

> If I'm not suffering from too low blood levels of caffeine right now,
> then from these 3 constraints follows:
> 
> - default streams are basically useless (since they cannot target
> multiple fedora releases in most cases, due to the Update Policy),

In general, yes. If the package versions have incompatibilities and/or
user-visible changes, a different stream is needed for each Fedora
release. There was a subthread about this recently, starting at
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/C5EX4WI7Z4HZZLVTUIHSMMPPUGTFENYE/.

> - flexible lifecycle advantages of modules do not apply to fedora,
> since module EOL dates must align with fedora release EOL dates.
> 
> Then, what *is* the benefit of using modules for "default" versions of
> fedora packages, if "default" streams have to usually be maintained
> separately for different fedora branches, just like normal packages,
> but with the *additional* overhead of Modularity - and additional work
> for maintainers of dependent packages?

That is one of questions we are trying to answer in this thread ;)

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Alexander Bokovoy]

On pe, 18 loka 2019, Martin Kolman wrote:

On Fri, 2019-10-18 at 11:39 +0300, Alexander Bokovoy wrote:

On to, 17 loka 2019, Orion Poplawski wrote:
> > > You could install the ipa-client package and enroll a system into IPA 
from a
> > > kickstart in RHEL 7 too.. Without modules. That's what I've deployed for 
the
> > > environments I support, for example. Using a module is not required there.
> >
> > That wasn't the point, though - the point was the answer the question
> > "why do we need *default* module streams?"
> >
> > The logic is this: FreeIPA maintainers wanted FreeIPA to be a module in
> > RHEL, to take advantage of the added flexibility around lifecycles and
> > version bumps (basically so each RHEL release isn't tied to one version
> > of FreeIPA forever). But if it's modularized and there's no concept of
> > 'default stream modules', this is a thing that breaks: you can't
> > install it from a kickstart. So, *given that* we wanted to modularize
> > FreeIPA in RHEL *and* we also want to still make it deployable via
> > kickstart, that creates a requirement for default stream modules or
> > something a lot like it.
>
> This doesn't seem quite true.  You couldn't install it with the same kickstart
> you used for EL7, but you could use the new module command or syntax in 
kickstart:
>
> module --name=NAME [--stream=STREAM]
Actually, you could install client packages with the same kickstart file
as for RHEL 7, that was one of uses for default profiles.

Server package installation from kickstart file is less of a worry
because we are running a different deployment process since switching to
domain level 1 and that implies you have to do client installation
first.

And at the time when all this was designed, kickstart had no support for
modularized installation. It has it now, of course.

Well, module installation vias kickstart has been supported since before 8.0 GA.
But I guess the design decisions have taken place before that.


Yes, well before that. In any case, one of bigger requirements we had
was to keep support for existing kickstart files that install RHEL IdM.
Changing them for using modular content for most common use case
(installation of IdM client) was seen as a compatibility break.

So yes, RHEL 8.x has support for enabling modules in the kickstart files
but it was not possible to preserve existing kickstart files that used
ipa-client package without enabling default module stream after RHEL IdM
was moved to module.

--
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Fabio Valentini]

On Sun, Oct 20, 2019 at 11:09 AM Zbigniew Jędrzejewski-Szmek
 wrote:
>
> On Wed, Oct 16, 2019 at 02:48:13PM -0400, Matthew Miller wrote:
> > On Wed, Oct 16, 2019 at 08:31:10AM -0400, Stephen Gallagher wrote:
> > > This is not true. It should be *possible* to have a fully modularized
> > > distribution, but that isn't a specific goal for Fedora or RHEL.
> >
> > Because this keeps coming up, we talked about this at the Fedora Council
> > meeting today. Our goals for modularity are:
> >   2. Those alternate streams should be able to have different lifecycles.
>
> Hmm, it sounds like the Council hasn't taken into account the constraints
> on lifecycle of modules that we have slowly discovered during the last
> two years, constraints that are now part of FESCo-approved policy.
>
> Essentially, modules in Fedora are only allowed to EOL at EOL of Fedora 
> release.
> And to preserve stability for users, a.k.a. following the Update Policy,
> modules should only change to new major version at Fedora releases.
> This is exactly the same as for "normal" rpms.
>
> The lifecycle of modules in Fedora must be the same as lifecycle of
> Fedora releases, so no "different lifecycle" is possible.

Ok, just to be sure that I understand this correctly:

- module EOL dates must align with fedora release EOL dates,
- Update Policy is the same for modules as for normal packages,
- major package updates can only occur at "release upgrade" time

If I'm not suffering from too low blood levels of caffeine right now,
then from these 3 constraints follows:

- default streams are basically useless (since they cannot target
multiple fedora releases in most cases, due to the Update Policy),
- flexible lifecycle advantages of modules do not apply to fedora,
since module EOL dates must align with fedora release EOL dates.

Then, what *is* the benefit of using modules for "default" versions of
fedora packages, if "default" streams have to usually be maintained
separately for different fedora branches, just like normal packages,
but with the *additional* overhead of Modularity - and additional work
for maintainers of dependent packages?

Fabio

> >   1. Users should have alternate streams of software available.
> >   3. Packaging an individual stream for multiple outputs should be easier
> >  than before.
>
> Those *are* useful goals, but they should not be tied to specific technology,
> we should only care about the end-result.
>
> Thus, please replace "Our goals for modularity are" with "What we hope
> to achieve with modularity" or even "Our goal is for users to be able to".
>
> Zbyszek
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Zbigniew Jędrzejewski-Szmek]

On Wed, Oct 16, 2019 at 02:48:13PM -0400, Matthew Miller wrote:
> On Wed, Oct 16, 2019 at 08:31:10AM -0400, Stephen Gallagher wrote:
> > This is not true. It should be *possible* to have a fully modularized
> > distribution, but that isn't a specific goal for Fedora or RHEL.
>
> Because this keeps coming up, we talked about this at the Fedora Council
> meeting today. Our goals for modularity are:
>   2. Those alternate streams should be able to have different lifecycles.
 
Hmm, it sounds like the Council hasn't taken into account the constraints
on lifecycle of modules that we have slowly discovered during the last
two years, constraints that are now part of FESCo-approved policy.

Essentially, modules in Fedora are only allowed to EOL at EOL of Fedora release.
And to preserve stability for users, a.k.a. following the Update Policy,
modules should only change to new major version at Fedora releases.
This is exactly the same as for "normal" rpms.

The lifecycle of modules in Fedora must be the same as lifecycle of
Fedora releases, so no "different lifecycle" is possible.

>   1. Users should have alternate streams of software available.
>   3. Packaging an individual stream for multiple outputs should be easier
>  than before.

Those *are* useful goals, but they should not be tied to specific technology,
we should only care about the end-result.

Thus, please replace "Our goals for modularity are" with "What we hope
to achieve with modularity" or even "Our goal is for users to be able to".

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Kevin Fenzi]

On Fri, Oct 18, 2019 at 10:42:54AM -0700, Howard Howell wrote:
> I just got a new computer, an Intel with Nvidia 2060 graphics card.  I
> could NOT get fedora to install or boot.  For the first time since

Do you recall any specifics? This is very unlikely to be related to
modularity or anything to do with this thread, but more likely an
installer or kernel bug.

> fedora 7 I am off of fedora.Wayland drove me nuts, the changes to the
> OS were less than perfect, some applications I used to run would not
> run, and on and on, the inability to install and boot Fedora was the
> last straw for me.  I may come back someday, because up until Fedora
> 28, things were pretty good.  Get back to that standard, and I will
> likely come back.  Meanwhile I will no longer recommend Fedora to my
> friends until things stabilize.Good luck, guys.  I wish you all the
> best.
> Les H

If you do get a chance to try Fedora 31 once it's out and report a bug
on the issue, it's more likely to be fixed up. 

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Howard Howell]

On Fri, 2019-10-18 at 13:05 +0200, Lukas Ruzicka wrote:
> 
> > Or, even better (or worse): Sombody installs GIMP via GNOME
> > Software,
> > 
> > and under the hood, dnf does its magic and installs gimp from the
> > 
> > module, which might depend on another, even non-default module,
> > etc.
> > 
> > But then, what will happen when that module is EOL, and the user
> > has
> > 
> > never even interacted with dnf, or modules? Will system upgrades
> > break
> > 
> > and prompt the user to fix something they didn't even know existed,
> > 
> > and have never interacted with?
> 
> This has already happened. We have had complaints from people who had
> neverinstalled any module, or used the "dnf module" command and they
> still ended up withmodules installed with a problem to upgrade their
> computers to F31.
> 
> 
>  
> > 
> > Fabio
> > 
> > 
> > 
> > > Pierre
> > 
> > > ___
I just got a new computer, an Intel with Nvidia 2060 graphics card.  I
could NOT get fedora to install or boot.  For the first time since
fedora 7 I am off of fedora.Wayland drove me nuts, the changes to the
OS were less than perfect, some applications I used to run would not
run, and on and on, the inability to install and boot Fedora was the
last straw for me.  I may come back someday, because up until Fedora
28, things were pretty good.  Get back to that standard, and I will
likely come back.  Meanwhile I will no longer recommend Fedora to my
friends until things stabilize.Good luck, guys.  I wish you all the
best.
Les H
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Matthew Miller]

On Fri, Oct 18, 2019 at 01:03:24PM +0200, Lukas Ruzicka wrote:
> Exactly ... this is what I believe, too. I think that Fedora users put
> Fedora on their desktops and laptops to be creative in many ways of
> creativity. Some make make music, some enhance pictures, some model in
> Blender, cut videos, write documents. The majority, I dare to say, is not
> interested in having several Inkscape versions, they want the newest yet
> stable enough and they are satisfied with that.

Well, maybe. Here's an actual Fedora story. A few releases ago, we had a
designer working on a little animation promoting how trouble-free and
painless Fedora updates are now (as a response to the "you should do an LTS,
or a rolling release!" messages I often here).

But, in the middle of making this, they updated themselves, and the new
version of Inkscape dropped support for a feature (a file format, I think)
that was very important to their work —  so they said that they couldn't
continue making that ad in good conscience.

If we had Inkscape as two streams, independent of the OS release, they could
have opted to continue with the one that worked for them for a while longer
at least.


-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Martin Kolman]

On Thu, 2019-10-17 at 14:44 -0600, Orion Poplawski wrote:
> On 10/17/19 2:35 PM, Adam Williamson wrote:
> > On Thu, 2019-10-17 at 09:32 -0700, John M. Harris Jr wrote:
> > > On Thursday, October 17, 2019 1:59:19 AM MST Alexander Bokovoy wrote:
> > > > The one thing we are using default modular stream in RHEL 8 for is to be
> > > > able to provide access to packages in kickstart that were moved to
> > > > modules in RHEL 8. An example is idm:client stream which is a default
> > > > module stream in RHEL 8 exactly for this reason, to be able to install
> > > > ipa-client package and enroll a system into IPA from a kickstart file.
> > > > 
> > > > We don't package FreeIPA in modules in Fedora yet but this is one of
> > > > real examples how default module streams are helpful to maintain
> > > > coherent user experience for existing users of kickstart files.
> > > > 
> > > > -- 
> > > > / Alexander Bokovoy
> > > > Sr. Principal Software Engineer
> > > > Security / Identity Management Engineering
> > > > Red Hat Limited, Finland
> > > 
> > > You could install the ipa-client package and enroll a system into IPA 
> > > from a 
> > > kickstart in RHEL 7 too.. Without modules. That's what I've deployed for 
> > > the 
> > > environments I support, for example. Using a module is not required there.
> > 
> > That wasn't the point, though - the point was the answer the question
> > "why do we need *default* module streams?"
> > 
> > The logic is this: FreeIPA maintainers wanted FreeIPA to be a module in
> > RHEL, to take advantage of the added flexibility around lifecycles and
> > version bumps (basically so each RHEL release isn't tied to one version
> > of FreeIPA forever). But if it's modularized and there's no concept of
> > 'default stream modules', this is a thing that breaks: you can't
> > install it from a kickstart. So, *given that* we wanted to modularize
> > FreeIPA in RHEL *and* we also want to still make it deployable via
> > kickstart, that creates a requirement for default stream modules or
> > something a lot like it.
> 
> This doesn't seem quite true.  You couldn't install it with the same kickstart
> you used for EL7, but you could use the new module command or syntax in 
> kickstart:
Indeed, you can install modules via kickstart. For details, see:
https://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#chapter-9-package-selection
https://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#module


> 
> module --name=NAME [--stream=STREAM]
This just enables a module stream (or can explicitely disable it with the 
--disable option). No packages
will be installed from such module unless specified in the %packages section.

> 
> and/or
> 
> %packages
> @module:stream/profile
This enables the module stream and installs a profile - the one specified or 
the default profile otherwise.

The syntax is pretty much the same as for DNF CLI - if you call "dnf install 
@module:stream/profile" you should
get the same result.


> 
> 
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Martin Kolman]

On Fri, 2019-10-18 at 11:39 +0300, Alexander Bokovoy wrote:
> On to, 17 loka 2019, Orion Poplawski wrote:
> > > > You could install the ipa-client package and enroll a system into IPA 
> > > > from a
> > > > kickstart in RHEL 7 too.. Without modules. That's what I've deployed 
> > > > for the
> > > > environments I support, for example. Using a module is not required 
> > > > there.
> > > 
> > > That wasn't the point, though - the point was the answer the question
> > > "why do we need *default* module streams?"
> > > 
> > > The logic is this: FreeIPA maintainers wanted FreeIPA to be a module in
> > > RHEL, to take advantage of the added flexibility around lifecycles and
> > > version bumps (basically so each RHEL release isn't tied to one version
> > > of FreeIPA forever). But if it's modularized and there's no concept of
> > > 'default stream modules', this is a thing that breaks: you can't
> > > install it from a kickstart. So, *given that* we wanted to modularize
> > > FreeIPA in RHEL *and* we also want to still make it deployable via
> > > kickstart, that creates a requirement for default stream modules or
> > > something a lot like it.
> > 
> > This doesn't seem quite true.  You couldn't install it with the same 
> > kickstart
> > you used for EL7, but you could use the new module command or syntax in 
> > kickstart:
> > 
> > module --name=NAME [--stream=STREAM]
> Actually, you could install client packages with the same kickstart file
> as for RHEL 7, that was one of uses for default profiles.
> 
> Server package installation from kickstart file is less of a worry
> because we are running a different deployment process since switching to
> domain level 1 and that implies you have to do client installation
> first.
> 
> And at the time when all this was designed, kickstart had no support for
> modularized installation. It has it now, of course.
Well, module installation vias kickstart has been supported since before 8.0 GA.
But I guess the design decisions have taken place before that.


> 
> -- 
> / Alexander Bokovoy
> Sr. Principal Software Engineer
> Security / Identity Management Engineering
> Red Hat Limited, Finland
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Pete Walter]

17.10.2019, 17:15, "Stephen John Smoogen" :
> On Wed, 16 Oct 2019 at 20:27, Stephen Gallagher  wrote:
>>
>
>>  So, literally every word of this is wrong. The negative feedback is
>>  not "overwhelming". It is approximately four noisy individuals, all of
>>  whom have expressed zero interest in understanding the actual
>>  situation that they are trying to "fix" by endlessly insulting the
>>  people working on the problem. Demoralizing the people who can dig us
>>  out of this situation is an unwise strategy.
>
> I realize you were tired and angry when reading this, but I want to
> say that I haven't put in my complaints because I find myself actually
> agreeing with the high level points pointed out by the 4 noisy
> individuals but find their method irritating and don't want to pile
> on.

Well said. I have also stayed quiet for much the same reasons, but I support 
what Kevin and Miro and others are saying.

It's definitely just not 4 noisy individuals. I'd even go as far as saying that 
most of the community here agrees with what they are saying.

Pete
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Lukas Ruzicka]

This does not work for server components and is not generalizable. For
> example, you cannot have multiple versions of Samba running on the same
> system. You cannot have multiple versions of FreeIPA running on the same
> system either. These server components have requirements beyond package
> installability.
>
> We have an answer for those use cases with VMs and containers and they
> aren't requiring parallel installability.
>
>
I am glad that this works for you.


I have been using Linux for 17 years as my only operating system since I
deleted the windows partitions from my desktop in
2002. I started with Mandrake, moving over Gentoo, to Archlinux and now
Fedora. And I am really proud that I can serve the
Fedora community.

In return, I would love to use Fedora as my primary (and my only) operating
system, because you use what you love and
what you love you want to make better. But if I cannot use it to run my
graphical, musical, and typesetting software nicely from
one single machine, I will not be able to use it primarily. Does it mean,
that I will have to keep at least two PCs, one for testing Fedora,
and another one for having fun? Most bugs I have noticed where coming from
using Fedora daily, not from looking into test environments.

I understand, that server people need stuff, but we, the desktop people,
need stuff, too. Let's try to make everyone happy. I know, we
can do it, if we want to.
But yeah, I fear the day, when I will have to put several containers
together to stitch myself a modern solution that any Linux distro provided
20 years ago.

I hope those days won't come.








> --
> / Alexander Bokovoy
> Sr. Principal Software Engineer
> Security / Identity Management Engineering
> Red Hat Limited, Finland
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>


-- 

Lukáš Růžička

FEDORA QE, RHCE

Red Hat



Purkyňova 115

612 45 Brno - Královo Pole

lruzi...@redhat.com
TRIED AND PERSONALLY TESTED, ERGO TRUSTED. 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Martin Kolman]

On Thu, 2019-10-17 at 09:47 -0400, Stephen Gallagher wrote:
> On Wed, Oct 16, 2019 at 9:39 PM Kevin Kofler  wrote:
> > So it looks like I did not describe clearly enough what my proposed
> > enable_modules=0 flag would do. ("Disable all module code" was apparently
> > too vague.)
> > 
> > How I think it should work would be:
> > * For repositories, it completely ignores modular metadata and processes
> >   only the non-modular parts of the repository metadata. Therefore, it does
> >   not see the Node.js 12.x stream at all. It only sees whatever Node.js is
> >   in the non-modular repository. If there are currently only modular
> >   versions, then it sees none at all. But with the proposal to require a
> >   non-modular default version, it would then see that version, which would
> >   likely be Node.js 10.x. Nobody would get forcefully upgraded from 10.x to
> >   12.x.
> > * For installed modules, it completely ignores them, acting as if the
> >   database of installed modules were empty. (It just does not read that
> >   database at all.)
> > * For installed packages, it treats them all as non-modular. Sure, packages
> >   originally installed from a module have weird EVRs encoding module
> >   metadata, but otherwise they get processed exactly like a non-modular
> >   package. So the default repository only has to provide a newer EVR to
> >   upgrade the package. That should address upgrades from 8.x or 10.x to the
> >   new default 10.x. If the user had previously installed 12.x, they will
> >   only get downgraded if they distro-sync or if package-level dependency
> >   issues with the F30 build of 12.x on F31 force the downgrade.
> > 
> > I hope that clears it up.
> 
> It does, thanks. I think you're right, that probably *would* work,
> though it's slightly harder to do your third bullet point than it
> sounds at first blush. I suppose we could fudge it with the
> `module_hotfix` option though..
> 
> > > > A slightly more elaborate, but slightly harder to implement, approach
> > > > would be to let DNF simply disable modules that are enabled locally but
> > > > no longer available in the repositories, together with disabling the
> > > > fedora-modular and updates-modular repositories by default.
> > > 
> > > And again, this only works if every packager who has spent time
> > > creating a module with a default stream takes their content and shoves
> > > it back into the non-modular repository. Which in some cases they
> > > probably cannot do, because they have build-dependencies that are in
> > > conflict. This is a highly non-trivial process and it would need to be
> > > done individually for every single package. That's far more
> > > packager-hostile than fixing the default stream/buildroot problem and
> > > the upgrade path problem.
> > 
> > You are overestimating by far the effort required to demodularize the
> > handful packages that are currently module-only. The evidence Fabio
> > Valentini has gathered so far shows that actually very few packages would be
> > affected and they would not be too hard to fix. And Miro has also offered
> > help with fixing affected packages.
> > 
> > All in all, it would require fixing a handful packages once and for all
> > instead of implementing workarounds affecting the entire distribution and
> > its thousands of users.
> > 
> 
> It's worth considering. I'm not ruling it out at this time. I'm not
> committed to doing it yet either.
> 
> > > > And the case of demodularizing packages has to be addressed sooner or
> > > > later anyway, so better address it sooner rather than later, before more
> > > > and more damage is done by maintainers moving packages to module-only
> > > > without a way back.
> > > 
> > > I've already acknowledged upthread that demodularizing packages is a
> > > problem we need to solve. It's being worked on, but it's a lot harder
> > > than you think, because we have failsafe code implemented in libdnf to
> > > prevent *accidental* demodularization that's in conflict with this
> > > desire.
> > 
> > And exactly that code needs to go, at least in Fedora. I think having a way
> > to migrate away from modules is the common case to prioritize here.
> > 
> 
> This is one of those cases where I think that RHEL and Fedora needs
> are in conflict; in RHEL, we absolutely need to support the failsafe
> behavior, because accidentally replacing a critical dependency will
> break user applications. In Fedora, this is likely a smaller concern.
> It needs investigation.
> 
> 
> > That said, it shall be pointed out that, if the proposal to demodularize all
> > default versions of packages gets implemented, we only need a *short term*
> > solution for demodularization in DNF. After 2 releases, we have no default
> > streams left (and they will never come back by policy) and we can expect
> > users to have upgraded through a release with no default streams (given that
> > we do not support upgrading directly to n+3), so at that point DNF can
> > revert 

Re: Modularity and the system-upgrade path

[Lukas Ruzicka]

Or, even better (or worse): Sombody installs GIMP via GNOME Software,
> and under the hood, dnf does its magic and installs gimp from the
> module, which might depend on another, even non-default module, etc.
> But then, what will happen when that module is EOL, and the user has
> never even interacted with dnf, or modules? Will system upgrades break
> and prompt the user to fix something they didn't even know existed,
> and have never interacted with?
>

This has already happened. We have had complaints from people who had never
installed any module, or used the "dnf module" command and they still ended
up with
modules installed with a problem to upgrade their computers to F31.




>
> Fabio
>
> > Pierre
> > ___
> > devel mailing list -- devel@lists.fedoraproject.org
> > To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> > Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>


-- 

Lukáš Růžička

FEDORA QE, RHCE

Red Hat



Purkyňova 115

612 45 Brno - Královo Pole

lruzi...@redhat.com
TRIED AND PERSONALLY TESTED, ERGO TRUSTED. 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Lukas Ruzicka]

>
>
> I'm comfortable saying that most Fedora users are not installing the
> distro
> just to support one specific application, as one might with RHEL or
> CentOS,
> but to benefit from the Four Foundations of Fedora, in this case the most
> important ones being Freedom, Features and First.
>

Exactly ... this is what I believe, too. I think that Fedora users put
Fedora on their
desktops and laptops to be creative in many ways of creativity. Some make
make music,
some enhance pictures, some model in Blender, cut videos, write documents.
The majority, I dare to say, is not interested in having several Inkscape
versions, they
want the newest yet stable enough and they are satisfied with that.


> It'd be great to have a working modular system, but since we don't seem to
> have that, it's not a good idea to force the broken implementation on
> users.
> We need to consider what is best for Fedora's users, not what is best for
> Red
> Hat, at least in my opinion.
>

Fedora modules must be ready to work in all possible combinations and
streams, if we really
mean it seriously. For example, I as a user, want to install the newest
version of Gimp, because I
need the newest features, but since the newest Scanner Application stopped
supporting my device,
I need the penultimate one. I also play windows games with wine and I set
the current version of
wine to suit my needs, so I want to stick with this version as long as
possible and maybe even beyond,
and I also want an NFS share for my TV to consume, but because I am
paranoid, I want to go 2 versions behind
the latest.

To make a long story short, I will need lots of different stream working in
harmony and I will want
to upgrade my PC without any problems.
Until we can provide this, we should keep modularity as opt-in technology
preview.



>
> I see no reason that dropping certain parts of Modularity from actual
> releases
> of Fedora will harm the relationship with Red Hat, as Stephen suggests.
> Such
> tests can, and probably should, be done in Rawhide, until they're actually
> ready for users.
>
> So far, the best approach seems to be to remove default modules, and
> require a
> non-modular version for fedora releases and branched. (In addition to
> whatever
> packagers would package as modules. To clarify, I am not attempting to
> suggest
> nothing should be done with Modularity except in Rawhide.)
>

This seems to me the easiest way to solve current problems.



>
> We're not saying this to discourage you, at least that is not my goal. My
> goal
> is to ensure the best result for the end user.
>
> --
> John M. Harris, Jr.
> Splentity
>
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>


-- 

Lukáš Růžička

FEDORA QE, RHCE

Red Hat



Purkyňova 115

612 45 Brno - Královo Pole

lruzi...@redhat.com
TRIED AND PERSONALLY TESTED, ERGO TRUSTED. 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Alexander Bokovoy]

On pe, 18 loka 2019, Kevin Kofler wrote:

Alexander Bokovoy wrote:

That's my point -- requiring parallel installability is not really a
MUST, especially in my area. You are driving this requirement as if
nothing else could solve your issues.


I am not. This is a strawman.

What I am saying is that modules on which other modules have versioned
dependencies cause version conflicts if they are not parallel-installable
(which is sadly the case now). There is no need for different versions of
leaf modules to be parallel-installable in most cases. (It could be useful
in some special cases, such as the game Battle for Wesnoth where savegames
are only usable with a specific release branch, but that is not the common
case. For the typical server application, I agree that it would be useless.)

ok, so your argument is for having module streams installable in
parallel for cases where they provide content that doesn't conflict on
rpm level.

I've been told this is not possible, so far, due to lack of right
metadata connection top to bottom between all layers involved (rpm,
libsolv, dnf, repositories, etc). I don't know how far that holds but I
would imagine this issue would hold for whatever technology is used to
aggregate sets of packages in the collection of repositories.

--
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Kevin Kofler]

Alexander Bokovoy wrote:
> That's my point -- requiring parallel installability is not really a
> MUST, especially in my area. You are driving this requirement as if
> nothing else could solve your issues.

I am not. This is a strawman.

What I am saying is that modules on which other modules have versioned 
dependencies cause version conflicts if they are not parallel-installable 
(which is sadly the case now). There is no need for different versions of 
leaf modules to be parallel-installable in most cases. (It could be useful 
in some special cases, such as the game Battle for Wesnoth where savegames 
are only usable with a specific release branch, but that is not the common 
case. For the typical server application, I agree that it would be useless.)

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Alexander Bokovoy]

On to, 17 loka 2019, Orion Poplawski wrote:

You could install the ipa-client package and enroll a system into IPA from a
kickstart in RHEL 7 too.. Without modules. That's what I've deployed for the
environments I support, for example. Using a module is not required there.


That wasn't the point, though - the point was the answer the question
"why do we need *default* module streams?"

The logic is this: FreeIPA maintainers wanted FreeIPA to be a module in
RHEL, to take advantage of the added flexibility around lifecycles and
version bumps (basically so each RHEL release isn't tied to one version
of FreeIPA forever). But if it's modularized and there's no concept of
'default stream modules', this is a thing that breaks: you can't
install it from a kickstart. So, *given that* we wanted to modularize
FreeIPA in RHEL *and* we also want to still make it deployable via
kickstart, that creates a requirement for default stream modules or
something a lot like it.


This doesn't seem quite true.  You couldn't install it with the same kickstart
you used for EL7, but you could use the new module command or syntax in 
kickstart:

module --name=NAME [--stream=STREAM]

Actually, you could install client packages with the same kickstart file
as for RHEL 7, that was one of uses for default profiles.

Server package installation from kickstart file is less of a worry
because we are running a different deployment process since switching to
domain level 1 and that implies you have to do client installation
first.

And at the time when all this was designed, kickstart had no support for
modularized installation. It has it now, of course.

--
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Alexander Bokovoy]

On pe, 18 loka 2019, Kevin Kofler wrote:

Alexander Bokovoy wrote:

This does not work for server components and is not generalizable. For
example, you cannot have multiple versions of Samba running on the same
system. You cannot have multiple versions of FreeIPA running on the same
system either. These server components have requirements beyond package
installability.


Technically, you can, on a different port. Of course, this kind of service
is probably more or less useless on a non-default port though.

But you would not be running multiple versions of the server at once. Why
would you want to do that? You would possibly parallel-install the client
libraries, if you have software linked to different versions of it, but why
the server?

That's my point -- requiring parallel installability is not really a
MUST, especially in my area. You are driving this requirement as if nothing
else could solve your issues.


Servers are typically pretty much leaf applications and as such can be
handled as any other leaf application, by shipping a default version in the
distribution and alternate versions in a module. Of course, if the server
links to the client library (e.g., MySQL and early versions of MariaDB used
to do that, before the separate MariaDB Connector/C was introduced), then
the module must include a version of the client library packaged in a way
that does not conflict with the system version that client applications are
linked to. But this can always be done.

Exactly, and this is what we do (in RHEL). We were thinking on making a
similar setup for Samba AD in modules in Fedora, but didn't go too far
because $TIME.




We have an answer for those use cases with VMs and containers and they
aren't requiring parallel installability.


Parallel installability of leaf software is not what I am proposing. It is
only needed for libraries.

Libraries I can understand.

--
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Kevin Kofler]

Adam Williamson wrote:
> Of course if you just don't modularize FreeIPA at all you don't have
> the kickstart problem, but then you *do* still have the 'we're stuck
> shipping this one version of FreeIPA for the next seventy jillion
> years' problem.

That is purely a RHEL thing though. I do not see how this is relevant to the 
discussion on whether to allow default streams *in Fedora*.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Kevin Kofler]

Stephen Gallagher wrote:
> I think that's a little harsh (but probably fair given my tone above).
> Can we agree that we're both on the same side: we want Fedora to be
> excellent?

I accept your apologies for your harsh tone (and I appreciate your much more 
constructive reply this time, thank you!) and I would like to apologize for 
my harsh tone as well. (I know I can be quite rude at times, especially when 
triggered.)

Yes, I agree that wanting Fedora to be excellent is probably what we all 
want. We may disagree about the way to get there, but let us sort this out 
constructively.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Kevin Kofler]

Przemek Klosowski via devel wrote:
>  3. modularity allows choosing non-default versions, which is great for
> a particular application, but conflicts with other apps, forcing us
> to choose only one of them. This provides a working solution for at
> least some people, so it's useful for e.g. Redhat, but it makes life
> hard for an end-user that just wants to have a system with a
> complete set of software

Exactly. And we already have a solution to that (allowing to choose non-
default versions of libraries without introducing this type of conflicts), 
it is called compatibility packages.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Kevin Kofler]

Alexander Bokovoy wrote:
> This does not work for server components and is not generalizable. For
> example, you cannot have multiple versions of Samba running on the same
> system. You cannot have multiple versions of FreeIPA running on the same
> system either. These server components have requirements beyond package
> installability.

Technically, you can, on a different port. Of course, this kind of service 
is probably more or less useless on a non-default port though.

But you would not be running multiple versions of the server at once. Why 
would you want to do that? You would possibly parallel-install the client 
libraries, if you have software linked to different versions of it, but why 
the server?

Servers are typically pretty much leaf applications and as such can be 
handled as any other leaf application, by shipping a default version in the 
distribution and alternate versions in a module. Of course, if the server 
links to the client library (e.g., MySQL and early versions of MariaDB used 
to do that, before the separate MariaDB Connector/C was introduced), then 
the module must include a version of the client library packaged in a way 
that does not conflict with the system version that client applications are 
linked to. But this can always be done.

> We have an answer for those use cases with VMs and containers and they
> aren't requiring parallel installability.

Parallel installability of leaf software is not what I am proposing. It is 
only needed for libraries.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Adam Williamson]

On Thu, 2019-10-17 at 14:44 -0600, Orion Poplawski wrote:
> On 10/17/19 2:35 PM, Adam Williamson wrote:
> > On Thu, 2019-10-17 at 09:32 -0700, John M. Harris Jr wrote:
> > > On Thursday, October 17, 2019 1:59:19 AM MST Alexander Bokovoy wrote:
> > > > The one thing we are using default modular stream in RHEL 8 for is to be
> > > > able to provide access to packages in kickstart that were moved to
> > > > modules in RHEL 8. An example is idm:client stream which is a default
> > > > module stream in RHEL 8 exactly for this reason, to be able to install
> > > > ipa-client package and enroll a system into IPA from a kickstart file.
> > > > 
> > > > We don't package FreeIPA in modules in Fedora yet but this is one of
> > > > real examples how default module streams are helpful to maintain
> > > > coherent user experience for existing users of kickstart files.
> > > > 
> > > > -- 
> > > > / Alexander Bokovoy
> > > > Sr. Principal Software Engineer
> > > > Security / Identity Management Engineering
> > > > Red Hat Limited, Finland
> > > 
> > > You could install the ipa-client package and enroll a system into IPA 
> > > from a 
> > > kickstart in RHEL 7 too.. Without modules. That's what I've deployed for 
> > > the 
> > > environments I support, for example. Using a module is not required there.
> > 
> > That wasn't the point, though - the point was the answer the question
> > "why do we need *default* module streams?"
> > 
> > The logic is this: FreeIPA maintainers wanted FreeIPA to be a module in
> > RHEL, to take advantage of the added flexibility around lifecycles and
> > version bumps (basically so each RHEL release isn't tied to one version
> > of FreeIPA forever). But if it's modularized and there's no concept of
> > 'default stream modules', this is a thing that breaks: you can't
> > install it from a kickstart. So, *given that* we wanted to modularize
> > FreeIPA in RHEL *and* we also want to still make it deployable via
> > kickstart, that creates a requirement for default stream modules or
> > something a lot like it.
> 
> This doesn't seem quite true.  You couldn't install it with the same kickstart
> you used for EL7, but you could use the new module command or syntax in 
> kickstart:
> 
> module --name=NAME [--stream=STREAM]
> 
> and/or
> 
> %packages
> @module:stream/profile

Hmm, yeah, I guess the concern is really about *existing* kickstarts.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Orion Poplawski]

On 10/17/19 2:35 PM, Adam Williamson wrote:
> On Thu, 2019-10-17 at 09:32 -0700, John M. Harris Jr wrote:
>> On Thursday, October 17, 2019 1:59:19 AM MST Alexander Bokovoy wrote:
>>> The one thing we are using default modular stream in RHEL 8 for is to be
>>> able to provide access to packages in kickstart that were moved to
>>> modules in RHEL 8. An example is idm:client stream which is a default
>>> module stream in RHEL 8 exactly for this reason, to be able to install
>>> ipa-client package and enroll a system into IPA from a kickstart file.
>>>
>>> We don't package FreeIPA in modules in Fedora yet but this is one of
>>> real examples how default module streams are helpful to maintain
>>> coherent user experience for existing users of kickstart files.
>>>
>>> -- 
>>> / Alexander Bokovoy
>>> Sr. Principal Software Engineer
>>> Security / Identity Management Engineering
>>> Red Hat Limited, Finland
>>
>> You could install the ipa-client package and enroll a system into IPA from a 
>> kickstart in RHEL 7 too.. Without modules. That's what I've deployed for the 
>> environments I support, for example. Using a module is not required there.
> 
> That wasn't the point, though - the point was the answer the question
> "why do we need *default* module streams?"
> 
> The logic is this: FreeIPA maintainers wanted FreeIPA to be a module in
> RHEL, to take advantage of the added flexibility around lifecycles and
> version bumps (basically so each RHEL release isn't tied to one version
> of FreeIPA forever). But if it's modularized and there's no concept of
> 'default stream modules', this is a thing that breaks: you can't
> install it from a kickstart. So, *given that* we wanted to modularize
> FreeIPA in RHEL *and* we also want to still make it deployable via
> kickstart, that creates a requirement for default stream modules or
> something a lot like it.

This doesn't seem quite true.  You couldn't install it with the same kickstart
you used for EL7, but you could use the new module command or syntax in 
kickstart:

module --name=NAME [--stream=STREAM]

and/or

%packages
@module:stream/profile


-- 
Orion Poplawski
Manager of NWRA Technical Systems  720-772-5637
NWRA, Boulder/CoRA Office FAX: 303-415-9702
3380 Mitchell Lane   or...@nwra.com
Boulder, CO 80301 https://www.nwra.com/



smime.p7s
Description: S/MIME Cryptographic Signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Adam Williamson]

On Thu, 2019-10-17 at 09:32 -0700, John M. Harris Jr wrote:
> On Thursday, October 17, 2019 1:59:19 AM MST Alexander Bokovoy wrote:
> > The one thing we are using default modular stream in RHEL 8 for is to be
> > able to provide access to packages in kickstart that were moved to
> > modules in RHEL 8. An example is idm:client stream which is a default
> > module stream in RHEL 8 exactly for this reason, to be able to install
> > ipa-client package and enroll a system into IPA from a kickstart file.
> > 
> > We don't package FreeIPA in modules in Fedora yet but this is one of
> > real examples how default module streams are helpful to maintain
> > coherent user experience for existing users of kickstart files.
> > 
> > -- 
> > / Alexander Bokovoy
> > Sr. Principal Software Engineer
> > Security / Identity Management Engineering
> > Red Hat Limited, Finland
> 
> You could install the ipa-client package and enroll a system into IPA from a 
> kickstart in RHEL 7 too.. Without modules. That's what I've deployed for the 
> environments I support, for example. Using a module is not required there.

That wasn't the point, though - the point was the answer the question
"why do we need *default* module streams?"

The logic is this: FreeIPA maintainers wanted FreeIPA to be a module in
RHEL, to take advantage of the added flexibility around lifecycles and
version bumps (basically so each RHEL release isn't tied to one version
of FreeIPA forever). But if it's modularized and there's no concept of
'default stream modules', this is a thing that breaks: you can't
install it from a kickstart. So, *given that* we wanted to modularize
FreeIPA in RHEL *and* we also want to still make it deployable via
kickstart, that creates a requirement for default stream modules or
something a lot like it.

Of course if you just don't modularize FreeIPA at all you don't have
the kickstart problem, but then you *do* still have the 'we're stuck
shipping this one version of FreeIPA for the next seventy jillion
years' problem.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Alexander Bokovoy]

On to, 17 loka 2019, Kevin Kofler wrote:

Dependencies aren't arbitrary; if they were, there would be probably no
need to waste our time in working on the whole build part. Whether that
is useful to you or other subset of Fedora maintainers is not
guaranteed. However, using modular streams allows to solve problems you
cannot easily solve otherwise within the same distribution for some use
cases. This is one part of value it brings that seems to be constantly
ignored with overly negative tone.

[snip]

Sure, for those things that can be installed in parallel. This is not
true for a wast amount of software, we have other means to deal with it
beyond what is being discussed in this thread.


Everything can be installed in parallel if appropriately packaged.

Having done the packaging tricks to allow kdelibs3-devel and kdelibs4-devel
to coexist (in the same /usr prefix, something upstream did not support), I
know exactly what I am talking about. (And for the next major version,
kf5-*-devel, we actually got upstream to care about this, so it is parallel-
installable with kdelibs3-devel and kdelibs4-devel out of the box. That is
really the ideal state to reach.)

This does not work for server components and is not generalizable. For
example, you cannot have multiple versions of Samba running on the same
system. You cannot have multiple versions of FreeIPA running on the same
system either. These server components have requirements beyond package
installability.

We have an answer for those use cases with VMs and containers and they
aren't requiring parallel installability.

--
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Przemek Klosowski via devel]

On 10/17/19 12:27 PM, Stephen John Smoogen wrote:

people are going to add things into their modules to make whatever
software they need. If I find that I need libfoo2-2.34 in libreoffice
and you need libfoo2-2.40 in evolution.. then only one of the two
modules can be installed.You can either have libreoffice or you can
have evolution.


Cap't Obvious here, but I think the logic is like this:

1. in an ideal world software would build and run with the
   latest-greatest versions of everything as a default
2. ...but in the real world we have to sometimes chose a non-default
   versions. There's enough of this happening that we can't just say
   we'll work hard until we reach 1.
3. modularity allows choosing non-default versions, which is great for
   a particular application, but conflicts with other apps, forcing us
   to choose only one of them. This provides a working solution for at
   least some people, so it's useful for e.g. Redhat, but it makes life
   hard for an end-user that just wants to have a system with a
   complete set of software
4. such modularized solutions can be combined into usable systems by
   either containers or cooperating VMs, but again, it's harder for
   end-users and has other undesirable consequences, e.g. complicates
   security management

The logical conundrum of modularity is that when we require non-default 
modules, then it logically follows that there will be conflicts (if 
there weren't, we wouldn't need modules) and so we are forced all the 
way into 4. unless we're lucky, and happen not to need the packages that 
depend on conflicting modules.


The bottom line is that modularity is useful, but in the sense of 
insurance or fire extinguishers: it's good to have them but we should 
really hope that we won't have to use them.


If only there was a way to limit the scope of the non-default modules to 
their dependencies--by using private library directories or something 
like that? I think it would solve the problem of parallel installation, 
and would simplify upgrades by making it explicit what pulled them in in 
the first place, and place joint responsibility for updates on these 
subsystems. This is essentially bundling, but exposed in the packaging 
system so it's more manageable.


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Kevin Kofler]

Alexander Bokovoy wrote:
> On to, 17 loka 2019, Kevin Kofler wrote:
>>Building against the distribution's version of libraries instead of some
>>arbitrarily picked version is pretty much the whole point of non-modular
>>packages.
> Right, and building against carefully chosen collection of dependencies
> is the whole point of modular packages. These are just two normal
> requirements that aren't contradicting each other most of the time.

Building against one shared distribution version of the library foo or 
building against a packager-chosen module stream version of the library foo 
are requirements that are very much contradicting each other by definition.

> Modular builds treat non-modular packages as a base environment to build
> on top. Sure, maintainers of modular streams need to take care of being
> non-conflicting on top of that, but sometimes the conflict is
> intentional as long as it is going to cover all dependencies broken by
> that. See, for example, some of scenarios in
> https://lists.centos.org/pipermail/centos-devel/2019-September/017774.html

Those are scenarios that are very specific to a long-term distribution such 
as RHEL or CentOS and do not commonly apply in Fedora.

In Fedora, you would typically ship a new FreeIPA in one of 2 ways:
1. as an official update to the existing Fedora release, if it is suitably
   compatible for that, OR
2. in the next Fedora release, which is, at any point in time, at most 6
   months away. Users who really cannot wait can get the update from a Copr.

And in fact, FreeIPA in Fedora is not currently a module, as you pointed out 
in your mail.

You would also likely not need to build against a newer krb5 than what 
Fedora ships. Or if you do, points 1 and 2 above also apply for krb5.

That whole "too fast, too slow" thing is really an issue specific to LTS 
distributions and not a pressing issue for a fast-moving distribution such 
as Fedora at all.

>>This is why building against arbitrary versions of non-leaf modules is a
>>recipe for version hell.
> You seem to be implying that whoever is maintaining a modular stream is
> not worth to trust that they are doing some reasonable work.

This is not a trust thing. No amount of "reasonable work" can prevent a 
module depending on libfoo-1 and a (from the user's point of view entirely 
unrelated) module depending on libfoo-2 from conflicting. The only 
"reasonable work" to do there is to package libfoo1 and libfoo2 as parallel-
installable packages (one of which will probably be called just libfoo, the 
other the suffixed name) instead of module streams to prevent the client 
applications from conflicting.

> Dependencies aren't arbitrary; if they were, there would be probably no
> need to waste our time in working on the whole build part. Whether that
> is useful to you or other subset of Fedora maintainers is not
> guaranteed. However, using modular streams allows to solve problems you
> cannot easily solve otherwise within the same distribution for some use
> cases. This is one part of value it brings that seems to be constantly
> ignored with overly negative tone.
[snip]
> Sure, for those things that can be installed in parallel. This is not
> true for a wast amount of software, we have other means to deal with it
> beyond what is being discussed in this thread.

Everything can be installed in parallel if appropriately packaged.

Having done the packaging tricks to allow kdelibs3-devel and kdelibs4-devel 
to coexist (in the same /usr prefix, something upstream did not support), I 
know exactly what I am talking about. (And for the next major version,
kf5-*-devel, we actually got upstream to care about this, so it is parallel-
installable with kdelibs3-devel and kdelibs4-devel out of the box. That is 
really the ideal state to reach.)

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Adam Williamson]

On Thu, 2019-10-17 at 13:43 +0200, Miro Hrončok wrote:
> On 17. 10. 19 13:38, Alexander Bokovoy wrote:
> > Had there be default module streams for Java packages in buildroot, we
> > would have no problem.
> 
> Had there been no default modular streams but regular packages instead, we 
> would 
> have no problem either.
> 
> But to extend there a bit, that would also be coorect had there been no 
> computers.

 Imagine no computers / Only farms of yaks... 
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[John M. Harris Jr]

On Thursday, October 17, 2019 1:59:19 AM MST Alexander Bokovoy wrote:
> The one thing we are using default modular stream in RHEL 8 for is to be
> able to provide access to packages in kickstart that were moved to
> modules in RHEL 8. An example is idm:client stream which is a default
> module stream in RHEL 8 exactly for this reason, to be able to install
> ipa-client package and enroll a system into IPA from a kickstart file.
> 
> We don't package FreeIPA in modules in Fedora yet but this is one of
> real examples how default module streams are helpful to maintain
> coherent user experience for existing users of kickstart files.
> 
> -- 
> / Alexander Bokovoy
> Sr. Principal Software Engineer
> Security / Identity Management Engineering
> Red Hat Limited, Finland

You could install the ipa-client package and enroll a system into IPA from a 
kickstart in RHEL 7 too.. Without modules. That's what I've deployed for the 
environments I support, for example. Using a module is not required there.

-- 
John M. Harris, Jr.
Splentity

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Stephen John Smoogen]

On Thu, 17 Oct 2019 at 10:06, Przemek Klosowski via devel
 wrote:
>
> On 10/16/19 7:36 PM, Kevin Kofler wrote:
>
> It was never designed to solve parallel installability problem.
>
> … which is exactly why it causes version hell.
>
> Could you expand on that? Since a modular system currently prevents parallel 
> version installation, it may provide suboptimal/obsolete versions, but I 
> wouldn't tap it as 'version hell', which in my mind describes a system with 
> multiple installed versions installed all over the place, causing confusion 
> and uncertainty as to the ABI compatibility and patching status.
>

[For the sake of this strawman... I am using libreoffice and evolution
which are only used because they are highly used items.. they could be
anylist of things like httpd and varnish or some other things you
would use together but aren't being modulared together.]

people are going to add things into their modules to make whatever
software they need. If I find that I need libfoo2-2.34 in libreoffice
and you need libfoo2-2.40 in evolution.. then only one of the two
modules can be installed. You can either have libreoffice or you can
have evolution. Furthermore whatever packages which were built against
the non-modular libfoo2-2.30 will either not work or be uninstalled
because the user decided they wanted libreoffice or evolution.

Each module 'owner' would have been making the best decision with the
time and energy they have to use modules for their problem: get the
latest software out for the most releases as easily as possible. They
will probably have tried to get libfoo2 updated in the core but found
that the libfoo2 maintainer is not responding or a CVE came out and it
had to be fixed now and the fix for libreoffice needed a version.

So after this happens the libreoffice and evolution decide to work
their modules together. That eventually turns into just one module ..
when they run into a problem with the firefox module which now is
pulling in a new libfoo2. They get pissed off and retire all of
libreoffice and evolution (and whatever else got pulled in as the
modular versions of X) or they combine with firefox.

Packages are like a super saturated liquid below the freezing point.
There are 20,000+ packages and ~400 active packagers. Little events
are going to cause either tiny crystals to grow around a package (a
module of 1 package like the RHEL perl-CGI) or a giant crystal (rust
and java are probably going to grow until anything built with those
languages will have to be in the module) and it will happen very very
fast.. must faster than expected.. and like a crystal growth it will
have lots of faults and crack open in ways not expected also.


> Perhaps you are saying that a hypothetical system allowing packaged parallel 
> installed versions provides the authoritative registry that tracks such 
> dependencies, and therefore does not have these problems?
>
>
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org



-- 
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Stephen John Smoogen]

On Wed, 16 Oct 2019 at 20:27, Stephen Gallagher  wrote:
>

> So, literally every word of this is wrong. The negative feedback is
> not "overwhelming". It is approximately four noisy individuals, all of
> whom have expressed zero interest in understanding the actual
> situation that they are trying to "fix" by endlessly insulting the
> people working on the problem. Demoralizing the people who can dig us
> out of this situation is an unwise strategy.
>

I realize you were tired and angry when reading this, but I want to
say that I haven't put in my complaints because I find myself actually
agreeing with the high level points pointed out by the 4 noisy
individuals but find their method irritating and don't want to pile
on.

My main problem with modularity is that it has been used to 'weld the
engine bonnet/hood shut'.  One of the catchy phrases from the early
years of Red Hat Linux was Bob Young's "Would you buy a car with the
hood welded shut?" It was something that attracted me to RHL and then
Fedora. I could get the things out of the OS and rebuild as I needed
without bugging others or having to go set up more than a small home
garage.

I find however that modularity is being used as a tool to weld parts
of the engine away and it drives me bonkers. I can't just take a bunch
of rpms from download.fedoraproject.org and rebuild them with some
options to get what I want. Instead I have to go dig into 'hidden'
parts of koji and other places to then figure out the secret
incantations used to stitch things together. Furthermore, I couldn't
just use my home garage to build things.. I can't use rpmbuild or
mock** to build a module set. Instead I was either supposed to send my
things to the Fedora koji factory and wait in line for a build.. or I
had to build an entire factory myself. The tools to let me experiment
and work on this in my 'garage' always seemed to be delivered as
afterthoughts which made me feel increasingly not wanted. [Again I
realize that this is non-rational.. but it is the non-rational things
which are driving this thread out..]

I have found myself in complete sympathy with the various independent
mechanics who can no longer work on various brands of cars because
they manufacturer decided to use only custom tools. Sure I could work
out and mill my own versions of those tools, but why should I when I
never 'asked' for this problem and I thought the manufacturer was
making cars that anyone could work on.


** I don't think it does yet, but msuchy (and team) may have added it
somewhere as a feature of --chain or something.




-- 
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Stephen Gallagher]

On Thu, Oct 17, 2019 at 10:17 AM Pierre-Yves Chibon  wrote:
>
> On Thu, Oct 17, 2019 at 03:38:39AM +0200, Kevin Kofler wrote:
> > So bringing yourselves up now as "the people who can dig us out of this
> > situation" feels to me feels like intentionally making a patient sick so you
> > can "cure" them.
>
> I can understand with the frustration you seem to have, but I think this 
> section
> is un-called for and not worth being sent to the devel list.
>
> Stephen is working on modularity either because he has been tasked to or 
> because
> he genuinely believe on the idea and goals,

It's both. I genuinely believe that when we get this right, it will be
a huge win for Fedora, RHEL and the rest of our expanded ecosystem.
The fact that Red Hat has seen fit to pay me to work on it is icing on
that cake.

> either way we want him involved in
> these discussions, otherwise any feedback anyone may have will never reached 
> the
> right person.
> As the project evolve some of the earlier assumptions turned to be correct and
> some turned to be wrong. I remember you raising some concerns early on, and
> there could be reasons for which they were not taken into account then.

I've been up-front about this: we were overconfident about some parts
of this and that has bitten us. I'm not denying the issues we face
today, but I *am* trying to make sure we make the right decisions for
the project long-term and not just the knee-jerk expedient ones.

> So let's not judge or try to re-write the past and focus on possible solutions
> as the rest of your email was doing.
> I'm not saying that your solution is the better one or the one that will be
> implemented, but having Stephen on this thread is the proof that it will at
> least be evaluated and thought through.
>
> Thank you Stephen for your involvement in this discussion.
>

Thank you, Pierre. I appreciate the vote of confidence.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Pierre-Yves Chibon]

On Thu, Oct 17, 2019 at 03:38:39AM +0200, Kevin Kofler wrote:
> > So, literally every word of this is wrong. The negative feedback is
> > not "overwhelming". It is approximately four noisy individuals, all of
> > whom have expressed zero interest in understanding the actual
> > situation that they are trying to "fix" by endlessly insulting the
> > people working on the problem. Demoralizing the people who can dig us
> > out of this situation is an unwise strategy.
> 
> LOL, filing off mailing list consensus as "four noisy individuals". This is 
> getting really ridiculous! Why am I even arguing with somebody who clearly 
> does not want to listen?
> 
> And you are the people who brought us into this situation to begin with. 
> Default streams should never have been allowed without:
> 1. an upgrade path from default stream to default stream, AND
> 2. a contingency plan, i.e., an upgrade path from default stream to
>non-modular default version
> The fact that they were implemented without EITHER of these (when actually 
> BOTH are needed) was extremely short-sighted.
> 
> So bringing yourselves up now as "the people who can dig us out of this 
> situation" feels to me feels like intentionally making a patient sick so you 
> can "cure" them.

I can understand with the frustration you seem to have, but I think this section
is un-called for and not worth being sent to the devel list.

Stephen is working on modularity either because he has been tasked to or because
he genuinely believe on the idea and goals, either way we want him involved in
these discussions, otherwise any feedback anyone may have will never reached the
right person.
As the project evolve some of the earlier assumptions turned to be correct and
some turned to be wrong. I remember you raising some concerns early on, and
there could be reasons for which they were not taken into account then.
So let's not judge or try to re-write the past and focus on possible solutions
as the rest of your email was doing.
I'm not saying that your solution is the better one or the one that will be
implemented, but having Stephen on this thread is the proof that it will at
least be evaluated and thought through.

Thank you Stephen for your involvement in this discussion.


Pierre
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Przemek Klosowski via devel]

On 10/16/19 7:36 PM, Kevin Kofler wrote:

It was never designed to solve parallel installability problem.

… which is exactly why it causes version hell.

Could you expand on that? Since a modular system currently prevents 
parallel version installation, it may provide suboptimal/obsolete 
versions, but I wouldn't tap it as 'version hell', which in my mind 
describes a system with multiple installed versions installed all over 
the place, causing confusion and uncertainty as to the ABI compatibility 
and patching status.


Perhaps you are saying that a hypothetical system allowing packaged 
parallel installed versions provides the authoritative registry that 
tracks such dependencies, and therefore does not have these problems?



___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Stephen John Smoogen]

On Thu, 17 Oct 2019 at 09:24, Miro Hrončok  wrote:
>
> On 17. 10. 19 15:17, Stephen Gallagher wrote:
> > On Thu, Oct 17, 2019 at 4:33 AM Miro Hrončok  wrote:
> >>
> >> On 17. 10. 19 2:27, Stephen Gallagher wrote:
> >>> So, literally every word of this is wrong. The negative feedback is
> >>> not "overwhelming". It is approximately four noisy individuals, all of
> >>> whom have expressed zero interest in understanding the actual
> >>> situation that they are trying to "fix" by endlessly insulting the
> >>> people working on the problem. Demoralizing the people who can dig us
> >>> out of this situation is an unwise strategy.
> >>
> >> Let me make this clear: I have a technical opinion (default modular 
> >> streams are
> >> the wrong thing to do). If we are not considering that opinion, because it
> >> insults the people who have implemented the technical thing, we are making 
> >> it
> >> personal.
> >>
> >> Everybody, please keep the discussion technical (that applies to both 
> >> "sides" here).
> >>
> >
> > Apologies for the tone here. That was out of line. I need to stop
> > replying when I'm tired. And I wasn't thinking of you, Miro, when I
> > wrote this. Your feedback hasn't been "negative", it's been
> > "constructive" (in the way I think of things).
>
> I appreciate you are trying to follow up on everything here. It must be very
> frustrating and I am sorry that my proposal has caused it.
>
> What bothers me ATM is that while the discussion is long and painful, it no
> longer moves anywhere :(
>
> IMHO Everybody have already said all their arguments at least twice. I wonder
> how to move forward.

When conversations loop like this, then the problem isn't what is
being stated.. it is the emotional problems which are being glossed
over. You can't just say 'let us keep this technical' because I expect
everyone thinks they are.. even when they aren't. Looking at the tone
and 'feelings' of what is being said, it looks like the '4 noisy
individuals' feel angry, and possibly betrayed and lied to.

They feel betrayed because several of them tried to point out that
pretty much every issue we ran into with libgit2, rust, java, and
other items were going to happen. And the answers they got were 'stop
impeding progress' to 'no people won't do those things because they
should know better', or 'if it happens we will come up with the
policies to make sure it doesn't again'. I am not saying they
expressed their concerns in a way that made people want to listen to
them, but the core of the issues of 'if you want to do this you need
to assume people are going to be jackasses 30% of the time, idiots 30%
of the time, do what is assumed 30%, and angels 10%. Write the
policies and tools to meet that.'

They feel lied to, because things have changed and the changes were
not what they expected. Maybe it is their expectations of what Fedora
was and what Fedora is  are different. To some we are dropping things
which they feel strongly attached to, and we are basically telling
them to move on or move out. If I don't want modularity.. I can't not
have it. If I want i386, I can't have it. If I want various packages
which were there before.. but are dead/gone.. I can't have it. If I
came here with an idea about an OS dedicated to Freedom, Friends,
First, and Features.. and found either that my Features are gone.. or
that I also have to share the OS with the projects sponsor's
decisions.. that all causes anger.

In any case, when things start cycling, a community needs to start
engaging in some sort of counseling to sort out the underlying
emotions which aren't getting addressed.



-- 
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Stephen Gallagher]

On Wed, Oct 16, 2019 at 9:39 PM Kevin Kofler  wrote:
>
> So it looks like I did not describe clearly enough what my proposed
> enable_modules=0 flag would do. ("Disable all module code" was apparently
> too vague.)
>
> How I think it should work would be:
> * For repositories, it completely ignores modular metadata and processes
>   only the non-modular parts of the repository metadata. Therefore, it does
>   not see the Node.js 12.x stream at all. It only sees whatever Node.js is
>   in the non-modular repository. If there are currently only modular
>   versions, then it sees none at all. But with the proposal to require a
>   non-modular default version, it would then see that version, which would
>   likely be Node.js 10.x. Nobody would get forcefully upgraded from 10.x to
>   12.x.
> * For installed modules, it completely ignores them, acting as if the
>   database of installed modules were empty. (It just does not read that
>   database at all.)
> * For installed packages, it treats them all as non-modular. Sure, packages
>   originally installed from a module have weird EVRs encoding module
>   metadata, but otherwise they get processed exactly like a non-modular
>   package. So the default repository only has to provide a newer EVR to
>   upgrade the package. That should address upgrades from 8.x or 10.x to the
>   new default 10.x. If the user had previously installed 12.x, they will
>   only get downgraded if they distro-sync or if package-level dependency
>   issues with the F30 build of 12.x on F31 force the downgrade.
>
> I hope that clears it up.

It does, thanks. I think you're right, that probably *would* work,
though it's slightly harder to do your third bullet point than it
sounds at first blush. I suppose we could fudge it with the
`module_hotfix` option though..

>
> >> A slightly more elaborate, but slightly harder to implement, approach
> >> would be to let DNF simply disable modules that are enabled locally but
> >> no longer available in the repositories, together with disabling the
> >> fedora-modular and updates-modular repositories by default.
> >
> > And again, this only works if every packager who has spent time
> > creating a module with a default stream takes their content and shoves
> > it back into the non-modular repository. Which in some cases they
> > probably cannot do, because they have build-dependencies that are in
> > conflict. This is a highly non-trivial process and it would need to be
> > done individually for every single package. That's far more
> > packager-hostile than fixing the default stream/buildroot problem and
> > the upgrade path problem.
>
> You are overestimating by far the effort required to demodularize the
> handful packages that are currently module-only. The evidence Fabio
> Valentini has gathered so far shows that actually very few packages would be
> affected and they would not be too hard to fix. And Miro has also offered
> help with fixing affected packages.
>
> All in all, it would require fixing a handful packages once and for all
> instead of implementing workarounds affecting the entire distribution and
> its thousands of users.
>

It's worth considering. I'm not ruling it out at this time. I'm not
committed to doing it yet either.

> >> And the case of demodularizing packages has to be addressed sooner or
> >> later anyway, so better address it sooner rather than later, before more
> >> and more damage is done by maintainers moving packages to module-only
> >> without a way back.
> >
> > I've already acknowledged upthread that demodularizing packages is a
> > problem we need to solve. It's being worked on, but it's a lot harder
> > than you think, because we have failsafe code implemented in libdnf to
> > prevent *accidental* demodularization that's in conflict with this
> > desire.
>
> And exactly that code needs to go, at least in Fedora. I think having a way
> to migrate away from modules is the common case to prioritize here.
>

This is one of those cases where I think that RHEL and Fedora needs
are in conflict; in RHEL, we absolutely need to support the failsafe
behavior, because accidentally replacing a critical dependency will
break user applications. In Fedora, this is likely a smaller concern.
It needs investigation.


> That said, it shall be pointed out that, if the proposal to demodularize all
> default versions of packages gets implemented, we only need a *short term*
> solution for demodularization in DNF. After 2 releases, we have no default
> streams left (and they will never come back by policy) and we can expect
> users to have upgraded through a release with no default streams (given that
> we do not support upgrading directly to n+3), so at that point DNF can
> revert to the "safe" behavior (preventing accidental demodularization) by
> default.
>

If we settle on the "no content in default streams" policy for Fedora,
this is a sensible way to go about it, yes.

> So the proposal to demodularize everything could 

Re: Modularity and the system-upgrade path

[Miro Hrončok]

On 17. 10. 19 15:17, Stephen Gallagher wrote:

On Thu, Oct 17, 2019 at 4:33 AM Miro Hrončok  wrote:


On 17. 10. 19 2:27, Stephen Gallagher wrote:

So, literally every word of this is wrong. The negative feedback is
not "overwhelming". It is approximately four noisy individuals, all of
whom have expressed zero interest in understanding the actual
situation that they are trying to "fix" by endlessly insulting the
people working on the problem. Demoralizing the people who can dig us
out of this situation is an unwise strategy.


Let me make this clear: I have a technical opinion (default modular streams are
the wrong thing to do). If we are not considering that opinion, because it
insults the people who have implemented the technical thing, we are making it
personal.

Everybody, please keep the discussion technical (that applies to both "sides" 
here).



Apologies for the tone here. That was out of line. I need to stop
replying when I'm tired. And I wasn't thinking of you, Miro, when I
wrote this. Your feedback hasn't been "negative", it's been
"constructive" (in the way I think of things).


I appreciate you are trying to follow up on everything here. It must be very 
frustrating and I am sorry that my proposal has caused it.


What bothers me ATM is that while the discussion is long and painful, it no 
longer moves anywhere :(


IMHO Everybody have already said all their arguments at least twice. I wonder 
how to move forward.


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Stephen Gallagher]

On Thu, Oct 17, 2019 at 4:33 AM Miro Hrončok  wrote:
>
> On 17. 10. 19 2:27, Stephen Gallagher wrote:
> > So, literally every word of this is wrong. The negative feedback is
> > not "overwhelming". It is approximately four noisy individuals, all of
> > whom have expressed zero interest in understanding the actual
> > situation that they are trying to "fix" by endlessly insulting the
> > people working on the problem. Demoralizing the people who can dig us
> > out of this situation is an unwise strategy.
>
> Let me make this clear: I have a technical opinion (default modular streams 
> are
> the wrong thing to do). If we are not considering that opinion, because it
> insults the people who have implemented the technical thing, we are making it
> personal.
>
> Everybody, please keep the discussion technical (that applies to both "sides" 
> here).
>

Apologies for the tone here. That was out of line. I need to stop
replying when I'm tired. And I wasn't thinking of you, Miro, when I
wrote this. Your feedback hasn't been "negative", it's been
"constructive" (in the way I think of things).
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Stephen John Smoogen]

On Wed, 16 Oct 2019 at 13:33, Stephen Gallagher  wrote:
>
> On Wed, Oct 16, 2019 at 1:19 PM Przemek Klosowski via devel
>  wrote:
> >
> > On 10/15/19 9:26 PM, Stephen Gallagher wrote:

> I'm saying that the policy should forbid the use of that feature
> except for an absolute emergency, requiring approval from FESCo or
> similar. It would exist for cases like "Oh crap, it turns out we've
> been shipping patented content in this stream and we're obligated to
> remove it" or something like that. It should never be used in the
> general case. Not even for "This is so old we should force upgrades".
> For that we should just drop the stream entirely from the next
> release, which would result in the upgrade being impossible until the
> user took a manual action to get off that stream.
>
> It would be my hope that the "obsoletes:" would never actually get
> used, but I'm generally in favor of planning for the worst case ahead
> of time if we can see it coming.

If there is one thing I have learned from watching modularity go
through multiple releases... anything you think shouldn't happen a lot
will. It is nothing new, when packagers learned about Epochs a long
time ago... they got used a lot also. We should not just plan for the
worst case where we need it, but we need to work on the policies
first. Trying to do the policies afterwords is what I think has caused
the most feelings of betrayal and anger which are coming up in various
people's emails.. especially when several of them pointed out that the
problems would occur and were told they were overblowing it.




-- 
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Modularity and the system-upgrade path

[Miro Hrončok]

On 17. 10. 19 13:38, Alexander Bokovoy wrote:

Had there be default module streams for Java packages in buildroot, we
would have no problem.


Had there been no default modular streams but regular packages instead, we would 
have no problem either.


But to extend there a bit, that would also be coorect had there been no 
computers.

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org