Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Stephen Gallagher]

On Thu, Nov 14, 2019 at 2:10 PM Miro Hrončok  wrote:
>
> On 14. 11. 19 19:39, Stephen Gallagher wrote:
> > On Thu, Nov 14, 2019 at 12:12 PM Miro Hrončok  wrote:
> >>
> >> On 09. 10. 19 22:46, Ben Cotton wrote:
> >>> https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
> >>>
> >>> Enable module default streams in the buildroot repository for modular
> >>> and non-modular RPMs.
> >>>
> >>> == Summary ==
> >>> This Change (colloquially referred to as "Ursa Prime") enables the
> >>> Koji build-system to include the RPM artifacts provided by module
> >>> default streams in the buildroot when building non-modular (or
> >>> "traditional") RPMs.
> >>
> >> I have one more technical concern.
> >>
> >> Suppose a packager decides to package the "mycoolapp" software as a 
> >> non-modular
> >> package. "mycoolapp" is written in Python, it builds again non-modular 
> >> Python,
> >> currently 3.8, it requires "python(abi) = 3.8" on runtime.
> >>
> >> The packager decides to use avocado in %check. Avocado comes from a 
> >> module, it
> >> requires "python(abi) = 3.8" as well, because the modular package was 
> >> built with
> >> Python 3.8. Avocado is in the bulidroot, so everything works.
> >>
> >> The Python maintainers (that includes me) decide to update to Python 3.9 in
> >> Fedora 33. They request a side tag to do that. They update the python3 to 
> >> 3.9
> >> and they mass rebuild all non-modular Python packages in it. "mycoolapp" 
> >> cannot
> >> resolve build dependencies because avocado requires "python(abi) = 3.8".
> >>
> >> The Python maintainers need to detect this and figure out what happened.
> >> Then, the Python maintainers need to either:
> >>
> >>1. Exclude "mycoolapp" from the rebuild. That is possible until dozens 
> >> of
> >> other packages require "mycoolapp".
> >>
> >>2. Ask the avocado maintainers to rebuild their module in the side tag 
> >> and ask
> >> releng to add the side-tag-built module into the side-tag buildroot (if it 
> >> is
> >> even possible).
> >>
> >>3. Modify the spec of "mycoolapp" to temporarily disable %check and 
> >> loose the
> >> avocado dependency.
> >>
> >> Or is there some other way?
> >>
> >
> > Does Python actually break ABI on minor releases? So a full rebuild is
> > required for that case?
>
> Yes, the full rebuild is needed for various reasons. We've talked about this 
> on
> IRC, but to make this more transparent to others, I will repeat it here. The
> main reasons include that "3.8" is in the paths (such as /usr/lib/python3.8) 
> and
> that the bytecode is changed. There are other reasons that may or may not be
> relevant to avocade.
>
> > What we would probably need to do is tag the python39 build into the
> > modular buildroot for the appropriate platform and then rebuild
> > avocado while that was in the buildroot.
>
> This would affect all modules that need to be rebuilt before the python39
> side-tag is merged, right?
>
> If I understand that right, it would also make the avocado module
> non-installable for regular rawhide buidlroot before the python39 side tag is
> merged.
>
> One of the purposes that we do this in the side tag is to avoid disrupting
> rawhide while we are mid-upgrade.
>
> > Another option would be to create a `python3` module stream for each
> > minor release and make one of them the default stream for the side-tag
> > so the non-modular versions could be built against that.
>
> Modular stream of what module?
>
> > Then the
> > avocado module would set its dependencies to be:
> > ```
> > buildrequires:
> >platform: [ ]
> >python3: [ ]
> > ```
> > (Read: build for all streams of python for all available platforms).
> > Then whenever the default stream of python3 changes over in Rawhide,
> > avocado would follow because it was already built for the next
> > version.
>
> Default stream of python3? python3 is not a module. And even if I wanted it to
> be (I don't), it would be a very hard thing to do.
>

So, what I meant here (as I said in another mail) was actually to
create a python3 module with streams for each release version whose
purpose is *not* to carry the actual python content. That should stay
with the non-modular RPMs. It would instead carry just a (new)
python-binaries package that would behave halfway between
`alternatives` and a metapackage, in that it would depend on the
appropriate non-modular packages and drop symlinks for the correct
binary versions into place. Then we could use those module streams as
source for doing stream expansion for modules that depend on Python.
Then it becomes easy to rebuild things like Avocado, since it will see
the new 3.9 stream and add that to the matrix.

Am I explaining that well?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: 

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Miro Hrončok]

On 14. 11. 19 20:58, Stephen Gallagher wrote:

On Thu, Nov 14, 2019 at 1:33 PM Miro Hrončok  wrote:


On 14. 11. 19 19:11, Zbigniew Jędrzejewski-Szmek wrote:

On Thu, Nov 14, 2019 at 01:00:52PM -0500, Neal Gompa wrote:

On Thu, Nov 14, 2019 at 12:59 PM Zbigniew Jędrzejewski-Szmek
 wrote:


On Thu, Nov 14, 2019 at 06:43:42PM +0100, Miro Hrončok wrote:

On 14. 11. 19 18:36, Zbigniew Jędrzejewski-Szmek wrote:

On Thu, Nov 14, 2019 at 06:08:52PM +0100, Miro Hrončok wrote:

On 09. 10. 19 22:46, Ben Cotton wrote:

https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot

Enable module default streams in the buildroot repository for modular
and non-modular RPMs.

== Summary ==
This Change (colloquially referred to as "Ursa Prime") enables the
Koji build-system to include the RPM artifacts provided by module
default streams in the buildroot when building non-modular (or
"traditional") RPMs.


I have one more technical concern.

Suppose a packager decides to package the "mycoolapp" software as a
non-modular package. "mycoolapp" is written in Python, it builds
again non-modular Python, currently 3.8, it requires "python(abi) =
3.8" on runtime.


Hmm, and what about an even simpler case:
"myswankyapp" is also written in Python, and is packaged as a module.
Python is rebuilt in a side tag, then the module blocks the upgrade.
What is supposed to happen in that case?


The module is rebuilt after the side tag is merged. Al least that is
what I think happened with avocado when we upgraded to Python 3.8.


Automatically? And if the build fails?



It's not automatic. Release Engineering has to kick it by hand by
making a weird empty commit and triggering a build.


And if the build fails?

In case this isn't obvious: right now python-sig will pummel all the
modules until they stop FTBFSing. If modules don't get the same
treatment, we have a problem.


Currently, modules don't get the same treatment. The reason basically is:

I don't know how to repoquery all modules to see what modular packages still
require Python 3.7. I was trying to get this information but at a certain point,
I've stopped trying.



That's a reasonable concern and a problem we do need to solve. Would
you mind adding it to https://pagure.io/modularity/issues ?


Done: https://pagure.io/modularity/issue/160

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Stephen Gallagher]

On Thu, Nov 14, 2019 at 1:33 PM Miro Hrončok  wrote:
>
> On 14. 11. 19 19:11, Zbigniew Jędrzejewski-Szmek wrote:
> > On Thu, Nov 14, 2019 at 01:00:52PM -0500, Neal Gompa wrote:
> >> On Thu, Nov 14, 2019 at 12:59 PM Zbigniew Jędrzejewski-Szmek
> >>  wrote:
> >>>
> >>> On Thu, Nov 14, 2019 at 06:43:42PM +0100, Miro Hrončok wrote:
>  On 14. 11. 19 18:36, Zbigniew Jędrzejewski-Szmek wrote:
> > On Thu, Nov 14, 2019 at 06:08:52PM +0100, Miro Hrončok wrote:
> >> On 09. 10. 19 22:46, Ben Cotton wrote:
> >>> https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
> >>>
> >>> Enable module default streams in the buildroot repository for modular
> >>> and non-modular RPMs.
> >>>
> >>> == Summary ==
> >>> This Change (colloquially referred to as "Ursa Prime") enables the
> >>> Koji build-system to include the RPM artifacts provided by module
> >>> default streams in the buildroot when building non-modular (or
> >>> "traditional") RPMs.
> >>
> >> I have one more technical concern.
> >>
> >> Suppose a packager decides to package the "mycoolapp" software as a
> >> non-modular package. "mycoolapp" is written in Python, it builds
> >> again non-modular Python, currently 3.8, it requires "python(abi) =
> >> 3.8" on runtime.
> >
> > Hmm, and what about an even simpler case:
> > "myswankyapp" is also written in Python, and is packaged as a module.
> > Python is rebuilt in a side tag, then the module blocks the upgrade.
> > What is supposed to happen in that case?
> 
>  The module is rebuilt after the side tag is merged. Al least that is
>  what I think happened with avocado when we upgraded to Python 3.8.
> >>>
> >>> Automatically? And if the build fails?
> >>>
> >>
> >> It's not automatic. Release Engineering has to kick it by hand by
> >> making a weird empty commit and triggering a build.
> >
> > And if the build fails?
> >
> > In case this isn't obvious: right now python-sig will pummel all the
> > modules until they stop FTBFSing. If modules don't get the same
> > treatment, we have a problem.
>
> Currently, modules don't get the same treatment. The reason basically is:
>
> I don't know how to repoquery all modules to see what modular packages still
> require Python 3.7. I was trying to get this information but at a certain 
> point,
> I've stopped trying.
>

That's a reasonable concern and a problem we do need to solve. Would
you mind adding it to https://pagure.io/modularity/issues ?

(Side note: that tracker has gotten badly backlogged, but Petr Sabata
and I are planning to go through it tomorrow to clean it up and
prioritize things. Don't be frightened by its current state!)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Miro Hrončok]

On 14. 11. 19 19:39, Stephen Gallagher wrote:

On Thu, Nov 14, 2019 at 12:12 PM Miro Hrončok  wrote:


On 09. 10. 19 22:46, Ben Cotton wrote:

https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot

Enable module default streams in the buildroot repository for modular
and non-modular RPMs.

== Summary ==
This Change (colloquially referred to as "Ursa Prime") enables the
Koji build-system to include the RPM artifacts provided by module
default streams in the buildroot when building non-modular (or
"traditional") RPMs.


I have one more technical concern.

Suppose a packager decides to package the "mycoolapp" software as a non-modular
package. "mycoolapp" is written in Python, it builds again non-modular Python,
currently 3.8, it requires "python(abi) = 3.8" on runtime.

The packager decides to use avocado in %check. Avocado comes from a module, it
requires "python(abi) = 3.8" as well, because the modular package was built with
Python 3.8. Avocado is in the bulidroot, so everything works.

The Python maintainers (that includes me) decide to update to Python 3.9 in
Fedora 33. They request a side tag to do that. They update the python3 to 3.9
and they mass rebuild all non-modular Python packages in it. "mycoolapp" cannot
resolve build dependencies because avocado requires "python(abi) = 3.8".

The Python maintainers need to detect this and figure out what happened.
Then, the Python maintainers need to either:

   1. Exclude "mycoolapp" from the rebuild. That is possible until dozens of
other packages require "mycoolapp".

   2. Ask the avocado maintainers to rebuild their module in the side tag and 
ask
releng to add the side-tag-built module into the side-tag buildroot (if it is
even possible).

   3. Modify the spec of "mycoolapp" to temporarily disable %check and loose the
avocado dependency.

Or is there some other way?



Does Python actually break ABI on minor releases? So a full rebuild is
required for that case?


Yes, the full rebuild is needed for various reasons. We've talked about this on 
IRC, but to make this more transparent to others, I will repeat it here. The 
main reasons include that "3.8" is in the paths (such as /usr/lib/python3.8) and 
that the bytecode is changed. There are other reasons that may or may not be 
relevant to avocade.



What we would probably need to do is tag the python39 build into the
modular buildroot for the appropriate platform and then rebuild
avocado while that was in the buildroot.


This would affect all modules that need to be rebuilt before the python39 
side-tag is merged, right?


If I understand that right, it would also make the avocado module 
non-installable for regular rawhide buidlroot before the python39 side tag is 
merged.


One of the purposes that we do this in the side tag is to avoid disrupting 
rawhide while we are mid-upgrade.



Another option would be to create a `python3` module stream for each
minor release and make one of them the default stream for the side-tag
so the non-modular versions could be built against that.


Modular stream of what module?


Then the
avocado module would set its dependencies to be:
```
buildrequires:
   platform: [ ]
   python3: [ ]
```
(Read: build for all streams of python for all available platforms).
Then whenever the default stream of python3 changes over in Rawhide,
avocado would follow because it was already built for the next
version.


Default stream of python3? python3 is not a module. And even if I wanted it to 
be (I don't), it would be a very hard thing to do.


See one failed attempt at https://fedoraproject.org/wiki/Changes/System_Python

See a very nasty hack in RHEL 8.

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Igor Gnatenko]

On Thu, Nov 14, 2019 at 7:48 PM Stephen Gallagher  wrote:
>
> On Thu, Nov 14, 2019 at 12:12 PM Miro Hrončok  wrote:
> >
> > On 09. 10. 19 22:46, Ben Cotton wrote:
> > > https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
> > >
> > > Enable module default streams in the buildroot repository for modular
> > > and non-modular RPMs.
> > >
> > > == Summary ==
> > > This Change (colloquially referred to as "Ursa Prime") enables the
> > > Koji build-system to include the RPM artifacts provided by module
> > > default streams in the buildroot when building non-modular (or
> > > "traditional") RPMs.
> >
> > I have one more technical concern.
> >
> > Suppose a packager decides to package the "mycoolapp" software as a 
> > non-modular
> > package. "mycoolapp" is written in Python, it builds again non-modular 
> > Python,
> > currently 3.8, it requires "python(abi) = 3.8" on runtime.
> >
> > The packager decides to use avocado in %check. Avocado comes from a module, 
> > it
> > requires "python(abi) = 3.8" as well, because the modular package was built 
> > with
> > Python 3.8. Avocado is in the bulidroot, so everything works.
> >
> > The Python maintainers (that includes me) decide to update to Python 3.9 in
> > Fedora 33. They request a side tag to do that. They update the python3 to 
> > 3.9
> > and they mass rebuild all non-modular Python packages in it. "mycoolapp" 
> > cannot
> > resolve build dependencies because avocado requires "python(abi) = 3.8".
> >
> > The Python maintainers need to detect this and figure out what happened.
> > Then, the Python maintainers need to either:
> >
> >   1. Exclude "mycoolapp" from the rebuild. That is possible until dozens of
> > other packages require "mycoolapp".
> >
> >   2. Ask the avocado maintainers to rebuild their module in the side tag 
> > and ask
> > releng to add the side-tag-built module into the side-tag buildroot (if it 
> > is
> > even possible).
> >
> >   3. Modify the spec of "mycoolapp" to temporarily disable %check and loose 
> > the
> > avocado dependency.
> >
> > Or is there some other way?
> >
>
> Does Python actually break ABI on minor releases? So a full rebuild is
> required for that case?
>
> What we would probably need to do is tag the python39 build into the
> modular buildroot for the appropriate platform and then rebuild
> avocado while that was in the buildroot.

That would mean, each release - bunch of additional manual work.

> Another option would be to create a `python3` module stream for each
> minor release and make one of them the default stream for the side-tag
> so the non-modular versions could be built against that. Then the
> avocado module would set its dependencies to be:
> ```
> buildrequires:
>   platform: [ ]
>   python3: [ ]
> ```
> (Read: build for all streams of python for all available platforms).
> Then whenever the default stream of python3 changes over in Rawhide,
> avocado would follow because it was already built for the next
> version.

This implies that Python maintainers want to deal with Modularity
which I believe is not the case.

> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Neal Gompa]

On Thu, Nov 14, 2019 at 1:40 PM Stephen Gallagher  wrote:
>
> On Thu, Nov 14, 2019 at 12:12 PM Miro Hrončok  wrote:
> >
> > On 09. 10. 19 22:46, Ben Cotton wrote:
> > > https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
> > >
> > > Enable module default streams in the buildroot repository for modular
> > > and non-modular RPMs.
> > >
> > > == Summary ==
> > > This Change (colloquially referred to as "Ursa Prime") enables the
> > > Koji build-system to include the RPM artifacts provided by module
> > > default streams in the buildroot when building non-modular (or
> > > "traditional") RPMs.
> >
> > I have one more technical concern.
> >
> > Suppose a packager decides to package the "mycoolapp" software as a 
> > non-modular
> > package. "mycoolapp" is written in Python, it builds again non-modular 
> > Python,
> > currently 3.8, it requires "python(abi) = 3.8" on runtime.
> >
> > The packager decides to use avocado in %check. Avocado comes from a module, 
> > it
> > requires "python(abi) = 3.8" as well, because the modular package was built 
> > with
> > Python 3.8. Avocado is in the bulidroot, so everything works.
> >
> > The Python maintainers (that includes me) decide to update to Python 3.9 in
> > Fedora 33. They request a side tag to do that. They update the python3 to 
> > 3.9
> > and they mass rebuild all non-modular Python packages in it. "mycoolapp" 
> > cannot
> > resolve build dependencies because avocado requires "python(abi) = 3.8".
> >
> > The Python maintainers need to detect this and figure out what happened.
> > Then, the Python maintainers need to either:
> >
> >   1. Exclude "mycoolapp" from the rebuild. That is possible until dozens of
> > other packages require "mycoolapp".
> >
> >   2. Ask the avocado maintainers to rebuild their module in the side tag 
> > and ask
> > releng to add the side-tag-built module into the side-tag buildroot (if it 
> > is
> > even possible).
> >
> >   3. Modify the spec of "mycoolapp" to temporarily disable %check and loose 
> > the
> > avocado dependency.
> >
> > Or is there some other way?
> >
>
> Does Python actually break ABI on minor releases? So a full rebuild is
> required for that case?
>

It is, yes. The Python ABI is versioned as MAJOR.MINOR.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Stephen Gallagher]

On Thu, Nov 14, 2019 at 12:12 PM Miro Hrončok  wrote:
>
> On 09. 10. 19 22:46, Ben Cotton wrote:
> > https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
> >
> > Enable module default streams in the buildroot repository for modular
> > and non-modular RPMs.
> >
> > == Summary ==
> > This Change (colloquially referred to as "Ursa Prime") enables the
> > Koji build-system to include the RPM artifacts provided by module
> > default streams in the buildroot when building non-modular (or
> > "traditional") RPMs.
>
> I have one more technical concern.
>
> Suppose a packager decides to package the "mycoolapp" software as a 
> non-modular
> package. "mycoolapp" is written in Python, it builds again non-modular Python,
> currently 3.8, it requires "python(abi) = 3.8" on runtime.
>
> The packager decides to use avocado in %check. Avocado comes from a module, it
> requires "python(abi) = 3.8" as well, because the modular package was built 
> with
> Python 3.8. Avocado is in the bulidroot, so everything works.
>
> The Python maintainers (that includes me) decide to update to Python 3.9 in
> Fedora 33. They request a side tag to do that. They update the python3 to 3.9
> and they mass rebuild all non-modular Python packages in it. "mycoolapp" 
> cannot
> resolve build dependencies because avocado requires "python(abi) = 3.8".
>
> The Python maintainers need to detect this and figure out what happened.
> Then, the Python maintainers need to either:
>
>   1. Exclude "mycoolapp" from the rebuild. That is possible until dozens of
> other packages require "mycoolapp".
>
>   2. Ask the avocado maintainers to rebuild their module in the side tag and 
> ask
> releng to add the side-tag-built module into the side-tag buildroot (if it is
> even possible).
>
>   3. Modify the spec of "mycoolapp" to temporarily disable %check and loose 
> the
> avocado dependency.
>
> Or is there some other way?
>

Does Python actually break ABI on minor releases? So a full rebuild is
required for that case?

What we would probably need to do is tag the python39 build into the
modular buildroot for the appropriate platform and then rebuild
avocado while that was in the buildroot.

Another option would be to create a `python3` module stream for each
minor release and make one of them the default stream for the side-tag
so the non-modular versions could be built against that. Then the
avocado module would set its dependencies to be:
```
buildrequires:
  platform: [ ]
  python3: [ ]
```
(Read: build for all streams of python for all available platforms).
Then whenever the default stream of python3 changes over in Rawhide,
avocado would follow because it was already built for the next
version.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Miro Hrončok]

On 14. 11. 19 19:11, Zbigniew Jędrzejewski-Szmek wrote:

On Thu, Nov 14, 2019 at 01:00:52PM -0500, Neal Gompa wrote:

On Thu, Nov 14, 2019 at 12:59 PM Zbigniew Jędrzejewski-Szmek
 wrote:


On Thu, Nov 14, 2019 at 06:43:42PM +0100, Miro Hrončok wrote:

On 14. 11. 19 18:36, Zbigniew Jędrzejewski-Szmek wrote:

On Thu, Nov 14, 2019 at 06:08:52PM +0100, Miro Hrončok wrote:

On 09. 10. 19 22:46, Ben Cotton wrote:

https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot

Enable module default streams in the buildroot repository for modular
and non-modular RPMs.

== Summary ==
This Change (colloquially referred to as "Ursa Prime") enables the
Koji build-system to include the RPM artifacts provided by module
default streams in the buildroot when building non-modular (or
"traditional") RPMs.


I have one more technical concern.

Suppose a packager decides to package the "mycoolapp" software as a
non-modular package. "mycoolapp" is written in Python, it builds
again non-modular Python, currently 3.8, it requires "python(abi) =
3.8" on runtime.


Hmm, and what about an even simpler case:
"myswankyapp" is also written in Python, and is packaged as a module.
Python is rebuilt in a side tag, then the module blocks the upgrade.
What is supposed to happen in that case?


The module is rebuilt after the side tag is merged. Al least that is
what I think happened with avocado when we upgraded to Python 3.8.


Automatically? And if the build fails?



It's not automatic. Release Engineering has to kick it by hand by
making a weird empty commit and triggering a build.


And if the build fails?

In case this isn't obvious: right now python-sig will pummel all the
modules until they stop FTBFSing. If modules don't get the same
treatment, we have a problem.


Currently, modules don't get the same treatment. The reason basically is:

I don't know how to repoquery all modules to see what modular packages still 
require Python 3.7. I was trying to get this information but at a certain point, 
I've stopped trying.


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Merlin Mathesius]

On Thu, Nov 14, 2019 at 12:09 PM Miro Hrončok  wrote:

> On 14. 11. 19 18:57, Zbigniew Jędrzejewski-Szmek wrote:
> > On Thu, Nov 14, 2019 at 06:43:42PM +0100, Miro Hrončok wrote:
> >> On 14. 11. 19 18:36, Zbigniew Jędrzejewski-Szmek wrote:
> >>> On Thu, Nov 14, 2019 at 06:08:52PM +0100, Miro Hrončok wrote:
>  On 09. 10. 19 22:46, Ben Cotton wrote:
> >
> https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
> >
> > Enable module default streams in the buildroot repository for modular
> > and non-modular RPMs.
> >
> > == Summary ==
> > This Change (colloquially referred to as "Ursa Prime") enables the
> > Koji build-system to include the RPM artifacts provided by module
> > default streams in the buildroot when building non-modular (or
> > "traditional") RPMs.
> 
>  I have one more technical concern.
> 
>  Suppose a packager decides to package the "mycoolapp" software as a
>  non-modular package. "mycoolapp" is written in Python, it builds
>  again non-modular Python, currently 3.8, it requires "python(abi) =
>  3.8" on runtime.
> >>>
> >>> Hmm, and what about an even simpler case:
> >>> "myswankyapp" is also written in Python, and is packaged as a module.
> >>> Python is rebuilt in a side tag, then the module blocks the upgrade.
> >>> What is supposed to happen in that case?
> >>
> >> The module is rebuilt after the side tag is merged. Al least that is
> >> what I think happened with avocado when we upgraded to Python 3.8.
> >
> > Automatically? And if the build fails?
>
> I don't think so. I think somebody actually went and did it. CCing Merlin.
>

 At least for avocado, yes, I manually rebuilt it.

Merlin
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Stephen Gallagher]

On Thu, Nov 14, 2019 at 11:52 AM Neal Gompa  wrote:
>
> > Similarly, from the perspective of dependent maintainers, there will
> > no longer be anxiety about needing to move their package to a module
> > if one or more of their dependencies drops their non-modular version
> > in favor of a default stream. Their builds will continue to work as
> > they do today.
> >
>
> This might be a dumb question (perhaps I missed the answer somewhere
> in the thread), but how do we handle setting overrides for modular
> content? Without that, it's going to be hard to do modular update to
> fix non-modular builds...

Sorry, I don't understand what question you're asking here. Could you
try using an example to make it clearer?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Zbigniew Jędrzejewski-Szmek]

On Thu, Nov 14, 2019 at 01:00:52PM -0500, Neal Gompa wrote:
> On Thu, Nov 14, 2019 at 12:59 PM Zbigniew Jędrzejewski-Szmek
>  wrote:
> >
> > On Thu, Nov 14, 2019 at 06:43:42PM +0100, Miro Hrončok wrote:
> > > On 14. 11. 19 18:36, Zbigniew Jędrzejewski-Szmek wrote:
> > > >On Thu, Nov 14, 2019 at 06:08:52PM +0100, Miro Hrončok wrote:
> > > >>On 09. 10. 19 22:46, Ben Cotton wrote:
> > > >>>https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
> > > >>>
> > > >>>Enable module default streams in the buildroot repository for modular
> > > >>>and non-modular RPMs.
> > > >>>
> > > >>>== Summary ==
> > > >>>This Change (colloquially referred to as "Ursa Prime") enables the
> > > >>>Koji build-system to include the RPM artifacts provided by module
> > > >>>default streams in the buildroot when building non-modular (or
> > > >>>"traditional") RPMs.
> > > >>
> > > >>I have one more technical concern.
> > > >>
> > > >>Suppose a packager decides to package the "mycoolapp" software as a
> > > >>non-modular package. "mycoolapp" is written in Python, it builds
> > > >>again non-modular Python, currently 3.8, it requires "python(abi) =
> > > >>3.8" on runtime.
> > > >
> > > >Hmm, and what about an even simpler case:
> > > >"myswankyapp" is also written in Python, and is packaged as a module.
> > > >Python is rebuilt in a side tag, then the module blocks the upgrade.
> > > >What is supposed to happen in that case?
> > >
> > > The module is rebuilt after the side tag is merged. Al least that is
> > > what I think happened with avocado when we upgraded to Python 3.8.
> >
> > Automatically? And if the build fails?
> >
> 
> It's not automatic. Release Engineering has to kick it by hand by
> making a weird empty commit and triggering a build.

And if the build fails?

In case this isn't obvious: right now python-sig will pummel all the
modules until they stop FTBFSing. If modules don't get the same
treatment, we have a problem.

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Miro Hrončok]

On 14. 11. 19 18:57, Zbigniew Jędrzejewski-Szmek wrote:

On Thu, Nov 14, 2019 at 06:43:42PM +0100, Miro Hrončok wrote:

On 14. 11. 19 18:36, Zbigniew Jędrzejewski-Szmek wrote:

On Thu, Nov 14, 2019 at 06:08:52PM +0100, Miro Hrončok wrote:

On 09. 10. 19 22:46, Ben Cotton wrote:

https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot

Enable module default streams in the buildroot repository for modular
and non-modular RPMs.

== Summary ==
This Change (colloquially referred to as "Ursa Prime") enables the
Koji build-system to include the RPM artifacts provided by module
default streams in the buildroot when building non-modular (or
"traditional") RPMs.


I have one more technical concern.

Suppose a packager decides to package the "mycoolapp" software as a
non-modular package. "mycoolapp" is written in Python, it builds
again non-modular Python, currently 3.8, it requires "python(abi) =
3.8" on runtime.


Hmm, and what about an even simpler case:
"myswankyapp" is also written in Python, and is packaged as a module.
Python is rebuilt in a side tag, then the module blocks the upgrade.
What is supposed to happen in that case?


The module is rebuilt after the side tag is merged. Al least that is
what I think happened with avocado when we upgraded to Python 3.8.


Automatically? And if the build fails?


I don't think so. I think somebody actually went and did it. CCing Merlin.

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Neal Gompa]

On Thu, Nov 14, 2019 at 12:59 PM Zbigniew Jędrzejewski-Szmek
 wrote:
>
> On Thu, Nov 14, 2019 at 06:43:42PM +0100, Miro Hrončok wrote:
> > On 14. 11. 19 18:36, Zbigniew Jędrzejewski-Szmek wrote:
> > >On Thu, Nov 14, 2019 at 06:08:52PM +0100, Miro Hrončok wrote:
> > >>On 09. 10. 19 22:46, Ben Cotton wrote:
> > >>>https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
> > >>>
> > >>>Enable module default streams in the buildroot repository for modular
> > >>>and non-modular RPMs.
> > >>>
> > >>>== Summary ==
> > >>>This Change (colloquially referred to as "Ursa Prime") enables the
> > >>>Koji build-system to include the RPM artifacts provided by module
> > >>>default streams in the buildroot when building non-modular (or
> > >>>"traditional") RPMs.
> > >>
> > >>I have one more technical concern.
> > >>
> > >>Suppose a packager decides to package the "mycoolapp" software as a
> > >>non-modular package. "mycoolapp" is written in Python, it builds
> > >>again non-modular Python, currently 3.8, it requires "python(abi) =
> > >>3.8" on runtime.
> > >
> > >Hmm, and what about an even simpler case:
> > >"myswankyapp" is also written in Python, and is packaged as a module.
> > >Python is rebuilt in a side tag, then the module blocks the upgrade.
> > >What is supposed to happen in that case?
> >
> > The module is rebuilt after the side tag is merged. Al least that is
> > what I think happened with avocado when we upgraded to Python 3.8.
>
> Automatically? And if the build fails?
>

It's not automatic. Release Engineering has to kick it by hand by
making a weird empty commit and triggering a build.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Zbigniew Jędrzejewski-Szmek]

On Thu, Nov 14, 2019 at 06:43:42PM +0100, Miro Hrončok wrote:
> On 14. 11. 19 18:36, Zbigniew Jędrzejewski-Szmek wrote:
> >On Thu, Nov 14, 2019 at 06:08:52PM +0100, Miro Hrončok wrote:
> >>On 09. 10. 19 22:46, Ben Cotton wrote:
> >>>https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
> >>>
> >>>Enable module default streams in the buildroot repository for modular
> >>>and non-modular RPMs.
> >>>
> >>>== Summary ==
> >>>This Change (colloquially referred to as "Ursa Prime") enables the
> >>>Koji build-system to include the RPM artifacts provided by module
> >>>default streams in the buildroot when building non-modular (or
> >>>"traditional") RPMs.
> >>
> >>I have one more technical concern.
> >>
> >>Suppose a packager decides to package the "mycoolapp" software as a
> >>non-modular package. "mycoolapp" is written in Python, it builds
> >>again non-modular Python, currently 3.8, it requires "python(abi) =
> >>3.8" on runtime.
> >
> >Hmm, and what about an even simpler case:
> >"myswankyapp" is also written in Python, and is packaged as a module.
> >Python is rebuilt in a side tag, then the module blocks the upgrade.
> >What is supposed to happen in that case?
> 
> The module is rebuilt after the side tag is merged. Al least that is
> what I think happened with avocado when we upgraded to Python 3.8.

Automatically? And if the build fails?

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Miro Hrončok]

On 14. 11. 19 18:36, Zbigniew Jędrzejewski-Szmek wrote:

On Thu, Nov 14, 2019 at 06:08:52PM +0100, Miro Hrončok wrote:

On 09. 10. 19 22:46, Ben Cotton wrote:

https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot

Enable module default streams in the buildroot repository for modular
and non-modular RPMs.

== Summary ==
This Change (colloquially referred to as "Ursa Prime") enables the
Koji build-system to include the RPM artifacts provided by module
default streams in the buildroot when building non-modular (or
"traditional") RPMs.


I have one more technical concern.

Suppose a packager decides to package the "mycoolapp" software as a
non-modular package. "mycoolapp" is written in Python, it builds
again non-modular Python, currently 3.8, it requires "python(abi) =
3.8" on runtime.


Hmm, and what about an even simpler case:
"myswankyapp" is also written in Python, and is packaged as a module.
Python is rebuilt in a side tag, then the module blocks the upgrade.
What is supposed to happen in that case?


The module is rebuilt after the side tag is merged. Al least that is what I 
think happened with avocado when we upgraded to Python 3.8.


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Zbigniew Jędrzejewski-Szmek]

On Thu, Nov 14, 2019 at 06:08:52PM +0100, Miro Hrončok wrote:
> On 09. 10. 19 22:46, Ben Cotton wrote:
> >https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
> >
> >Enable module default streams in the buildroot repository for modular
> >and non-modular RPMs.
> >
> >== Summary ==
> >This Change (colloquially referred to as "Ursa Prime") enables the
> >Koji build-system to include the RPM artifacts provided by module
> >default streams in the buildroot when building non-modular (or
> >"traditional") RPMs.
> 
> I have one more technical concern.
> 
> Suppose a packager decides to package the "mycoolapp" software as a
> non-modular package. "mycoolapp" is written in Python, it builds
> again non-modular Python, currently 3.8, it requires "python(abi) =
> 3.8" on runtime.

Hmm, and what about an even simpler case:
"myswankyapp" is also written in Python, and is packaged as a module.
Python is rebuilt in a side tag, then the module blocks the upgrade.
What is supposed to happen in that case?

Zbyszek

> The packager decides to use avocado in %check. Avocado comes from a
> module, it requires "python(abi) = 3.8" as well, because the modular
> package was built with Python 3.8. Avocado is in the bulidroot, so
> everything works.
> 
> The Python maintainers (that includes me) decide to update to Python
> 3.9 in Fedora 33. They request a side tag to do that. They update
> the python3 to 3.9 and they mass rebuild all non-modular Python
> packages in it. "mycoolapp" cannot resolve build dependencies
> because avocado requires "python(abi) = 3.8".
> 
> The Python maintainers need to detect this and figure out what happened.
> Then, the Python maintainers need to either:
> 
>  1. Exclude "mycoolapp" from the rebuild. That is possible until
> dozens of other packages require "mycoolapp".
> 
>  2. Ask the avocado maintainers to rebuild their module in the side
> tag and ask releng to add the side-tag-built module into the
> side-tag buildroot (if it is even possible).
> 
>  3. Modify the spec of "mycoolapp" to temporarily disable %check and
> loose the avocado dependency.
> 
> Or is there some other way?
> 
> -- 
> Miro Hrončok
> --
> Phone: +420777974800
> IRC: mhroncok
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Miro Hrončok]

On 09. 10. 19 22:46, Ben Cotton wrote:

https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot

Enable module default streams in the buildroot repository for modular
and non-modular RPMs.

== Summary ==
This Change (colloquially referred to as "Ursa Prime") enables the
Koji build-system to include the RPM artifacts provided by module
default streams in the buildroot when building non-modular (or
"traditional") RPMs.


I have one more technical concern.

Suppose a packager decides to package the "mycoolapp" software as a non-modular 
package. "mycoolapp" is written in Python, it builds again non-modular Python, 
currently 3.8, it requires "python(abi) = 3.8" on runtime.


The packager decides to use avocado in %check. Avocado comes from a module, it 
requires "python(abi) = 3.8" as well, because the modular package was built with 
Python 3.8. Avocado is in the bulidroot, so everything works.


The Python maintainers (that includes me) decide to update to Python 3.9 in 
Fedora 33. They request a side tag to do that. They update the python3 to 3.9 
and they mass rebuild all non-modular Python packages in it. "mycoolapp" cannot 
resolve build dependencies because avocado requires "python(abi) = 3.8".


The Python maintainers need to detect this and figure out what happened.
Then, the Python maintainers need to either:

 1. Exclude "mycoolapp" from the rebuild. That is possible until dozens of 
other packages require "mycoolapp".


 2. Ask the avocado maintainers to rebuild their module in the side tag and ask 
releng to add the side-tag-built module into the side-tag buildroot (if it is 
even possible).


 3. Modify the spec of "mycoolapp" to temporarily disable %check and loose the 
avocado dependency.


Or is there some other way?

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Neal Gompa]

On Wed, Oct 9, 2019 at 4:47 PM Ben Cotton  wrote:
>
> https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
>
> Enable module default streams in the buildroot repository for modular
> and non-modular RPMs.
>
> == Summary ==
> This Change (colloquially referred to as "Ursa Prime") enables the
> Koji build-system to include the RPM artifacts provided by module
> default streams in the buildroot when building non-modular (or
> "traditional") RPMs.
>
> == Owner ==
> * Name: [[User:Sgallagh| Stephen Gallagher]]
> * Email: sgall...@redhat.com
> * Responsible WG: Modularity WG
>
> == Detailed Description ==
> As a major part of the Modularity design, we have a concept of default
> module streams. These streams are built as modules, but the RPM
> artifacts they deliver are intended to be used just like non-modular
> RPMS. The aspirational goal is that a user of the system who never
> executes a module-specific command (such as `dnf module install
> nodejs:8`) should experience no meaningful changes in behavior from
> how they would interact with a completely non-modular system. In
> practice, this may mean that the informational output of package
> managers may indicate that modules are being enabled and used, but a
> user that does not have a specific reason to interact with the
> selection of a module stream should have that managed on their behalf.
>
> Similarly, the experience for package maintainers of non-modular
> packages should be unaffected by an RPM dependency moving from the
> non-modular repository into a default module stream. Up to the
> present, this has not been the case; no module stream content has been
> available in the non-modular buildroot for other packages to consume.
> Koji builds of non-modular RPMs have had only the other non-modular
> RPMs from that release available to their buildroots. In contrast,
> building on local systems has access to both the non-modular RPMs and
> the RPMs from any of the default module streams. With this Change,
> Koji builds will have the same behavior and be able to depend on
> content provided by default module streams. It also enables the same
> behavior for Modular builds: the `platform` stream will now include
> the contents of the default module streams for each release and do not
> need to be explicitly specified in the modulemd `buildrequires`.
>
> Note: This Change does not address the other major Modularity issue we
> are facing around distribution upgrades with differing default
> streams. When discussing this Change, please keep that topic separate.
>
> == Benefit to Fedora ==
>
> This will simplify the lives of package maintainers in Fedora in two
> primary ways. I'll use a hypothetical example of the Node.js
> interpreter and a JSApp package which is capable of running on Node.js
> 10 or 12 (but requires newer features than are provided by Node.js 8).
> Additionally, the JSApp package requires the same versions of Node.js
> at build-time.
>
> * Fedora 29 ships `nodejs:8`, `nodejs:10` and `nodejs:12` module
> streams. The `nodejs:10` stream is set as the default stream.
> * Fedora 30 ships `nodejs:8`, `nodejs:10` and `nodejs:12` module
> streams. The `nodejs:10` stream is set as the default stream.
> * Fedora 31 ships `nodejs:10` and `nodejs:12` module streams. The
> `nodejs:12` stream is set as the default stream. The `nodejs:14`
> stream will likely become available during the F31 lifetime.
> * Fedora 32 ships `nodejs:10` and `nodejs:12` module streams. The
> `nodejs:12` stream is set as the default stream. The `nodejs:14`
> stream will likely become available during the F32 lifetime.
>
> On Fedora 29 through 31, the Node.js package maintainer needs to build
> the `nodejs:10` package both as a module and as a non-modular RPM in
> the distribution so that the JSApp package can be built. With this
> Change, the Node.js package maintainer in Fedora 32+ will only need to
> build the various Node.js streams and make one of them the default
> stream. The packages from it will then be added to the buildroot for
> non-modular packages. This will also make the packaging process
> somewhat more efficient, as the maintainer needs only to manage the
> module stream and the MBS will build it for all configured platforms.
>
> Similarly, from the perspective of dependent maintainers, there will
> no longer be anxiety about needing to move their package to a module
> if one or more of their dependencies drops their non-modular version
> in favor of a default stream. Their builds will continue to work as
> they do today.
>

This might be a dumb question (perhaps I missed the answer somewhere
in the thread), but how do we handle setting overrides for modular
content? Without that, it's going to be hard to do modular update to
fix non-modular builds...



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to 

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Pierre-Yves Chibon]

On Wed, Nov 13, 2019 at 04:15:57PM -0500, Stephen Gallagher wrote:
> On Wed, Nov 13, 2019 at 1:24 PM Miro Hrončok  wrote:
> >
> > On 13. 11. 19 18:31, Stephen Gallagher wrote:
> > > Sorry for the long overdue reply here. Answers to your questions are 
> > > inline.
> >
> > Thank you.
> >
> > > On Wed, Oct 9, 2019 at 6:46 PM Miro Hrončok  wrote:
> > > ...
> > >> What I miss in the description is:
> > >>
> > >> 1. How does this thing actually work? is there an additional repository 
> > >> composed
> > >> from the default streams available in Koji only?
> > >
> > > An additional repository is composed from the default streams
> > > (including the module metadata) and is merged with the standard
> > > buildroot using Koji's new "bare" mode. This buildroot will be
> > > available for non-Koji purposes (such as mock).
> >
> > Will this buildroot repository be mirrored or will it be more like the 
> > current
> > "local" repository defined in mock configs?
> >
> > For reference, this is currently in the rawhide mock config:
> >
> > [local]
> > name=local
> > baseurl=https://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/
> > cost=2000
> > enabled=0
> > skip_if_unavailable=False
> >
> > My concern is that using the "local" repository for mock builds is quite 
> > slow
> > and IIRC puts Koji under unnecessary load.
> >
> 
> The plan for the moment is to have it be the "local" repository. Given
> that it will at least start out fairly small (and not need deltarpms),
> we were going to see how well it held up. The longer-term plan will
> probably be to mirror it.

Just a quick note, if you foresee some changes needed on MirrorManager to detect
and mirror this new repo you may want to reach to the MirrorManager maintainer
early enough so he can plan for them.


Pierre
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Stephen Gallagher]

On Wed, Nov 13, 2019 at 5:46 PM Miro Hrončok  wrote:
>
> On 13. 11. 19 22:15, Stephen Gallagher wrote:
...
> So the buildroot enabled modular repository contains only the 2 approved 
> modules
> including their modular metadata...
>
> >> "an enabled or default module"? Who enables a module in the buildroot? Is 
> >> it
> >> releng? The packager? Somebody else? Is it an explicit or implicit action?
> >>
> >
> > "enabled" could happen as part of a mock config. That would be an
> > explicit action on the part of the person creating the mock config.
> > For the purposes of the discussion of Ursa Prime, you can ignore
> > "enabled". I was trying to be thorough and ended up being confusing
> > instead.
>
> ...and in Koji mock config, both are enabled and/or default?

They will be marked as default in the modular-buildroot, yes.

>
> >> Suppose we put maven:3.5 into the buildroot (I know we don't do that, but 
> >> that
> >> was the idea origianlly). How do I as a a maintainer of an ursine maven-foo
> >> package BuildRequire the ursine maven package?
> >>
> >
> > You don't.
>  >
> > Making a stream into the default stream is effectively making it
> > equivalent to a non-modular package for most purposes. It already
> > behaves this way for client installations; the content from a default
> > stream is the only version DNF "sees" if it happens to override a
> > non-modular package. On the client system, you'd need to `dnf module
> > disable maven` to be able to pull down the non-modular package.
>
> I wonder what is the plan to handle this.
>
> I see only two proper ways out of here:
>
> 1) We only allow default modular streams if they only contain packages that 
> are
> retired from non-modular Fedora. Any request fro a default stream must be
> coordinated with all the relevant package maintainers and those packages must 
> be
> retired from Fedora.
>
> 2) No default modular streams. (Yes, that again.)
>
> In my opinion, if we allow the non-modular packages to stay when there is a
> default modular streams that "overrides" them, while effectively, our
> maintainers cannot use the non-modular packages in the buildroot and our users
> can only install the non-modular packages if they know how to disable a 
> module,
> it creates a very confusing situation.
>

You are correct. 1) is actually the policy[1] in Fedora today. There
are a couple modules that are in violation of the policy, which is the
cause of many of the issues we've been dealing with. The origin

Specifically: "[I]f the module stream masks part of the Traditional
RPM repos (e.g it replaces an existing RPM or it introduces a
non-trivial set of conflicts) it may not be made a default stream
without the express permission of FESCo. Unfortunately, the maven and
ant module streams were made default before this policy (and the
stronger one describing the api requirements) was in place. If this
had been submitted to FESCo today, it would have been refused.

[1] 
https://docs.fedoraproject.org/en-US/modularity/making-modules/managing-defaults/

> > Ursa Prime is basically a Change to have the Koji build environment
> > behave the same way as the local environment would.
> >
> > This is why we've been discussing the need for the artifacts of
> > default streams to behave *exactly* as if they were non-modular once
> > delivered. Because they effectively are. They were built differently
> > (though still in our infrastructure), but once built the requirement
> > is that they must then behave like the non-modular distro would have.
>
> I hope we agree that they currently don't behave exactly like non-modular.
>

Correct, we've beaten that horse already and identified the key pieces
that need to be fixed.

> > (Side-note: We retired the term "ursine" and replaced it with
> > "non-modular" because it was confusing for non-native English
> > speakers.)
>
> Excellent choice. It was indeed rather confusing. Thank you.
>
> Side-side-note: In my opinion, "Ursa Major/Prime" is equally confusing.
>

You're not wrong. But we needed some way to refer to "The project for
enabling the use of module artifacts in the non-modular buildroot"
that didn't take fifteen seconds to say every time. These are
essentially code-names, though. An end-user should never hear about
them and once it is in place and actually works, packagers should not
have to hear about it either.

>  3. What is the local experience if the packager is using mock. What if 
>  they are
>  using rpmbuild directly?
> >>>
> >>> For mock, if it's pointed at the Koji-hosted buildroot repository, it
> >>> will work exactly as Koji does (with no local changes required).
> >>
> >> Makes sense, however, the information I miss is: Will it be pointed to the
> >> Koji-hosted buildroot repository by default or not? My point was that this
> >> should be part of the proposal before it was approved.
> >>
> >
> > Yes, and you're right I will need to update the Change.
> >
> >>> If
> >>> it's pointed at the public 

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Miro Hrončok]

On 13. 11. 19 22:15, Stephen Gallagher wrote:

On Wed, Nov 13, 2019 at 1:24 PM Miro Hrončok  wrote:


On 13. 11. 19 18:31, Stephen Gallagher wrote:

Sorry for the long overdue reply here. Answers to your questions are inline.


Thank you.


On Wed, Oct 9, 2019 at 6:46 PM Miro Hrončok  wrote:
...

What I miss in the description is:

1. How does this thing actually work? is there an additional repository composed
from the default streams available in Koji only?


An additional repository is composed from the default streams
(including the module metadata) and is merged with the standard
buildroot using Koji's new "bare" mode. This buildroot will be
available for non-Koji purposes (such as mock).


Will this buildroot repository be mirrored or will it be more like the current
"local" repository defined in mock configs?

My concern is that using the "local" repository for mock builds is quite slow
and IIRC puts Koji under unnecessary load.



The plan for the moment is to have it be the "local" repository. Given
that it will at least start out fairly small (and not need deltarpms),
we were going to see how well it held up. The longer-term plan will
probably be to mirror it.


OK. Since currently, nothing really depends on the modules added there, it makes 
very little difference. I'm glad that there is a plan for the future.



2. How are conflicts between packages from the default streams and ursine
package be handled?


As long as the module metadata is present, an artifact with a given
name from an enabled or default module will effectively "hide" the
non-modular version.


I have a hard time to understand this. Could you please dive into the details?

"As long"? What does that mean? When they are not present and when they are 
present?



Sorry, that was an English colloquialism. Basically I meant "unless we
screw up and don't sync the metadata to the repository". So as long as
a module stream providing an artifact with a given name is enabled or
default, non-modular RPMs of the same name will be invisible to the
solver.


So the buildroot enabled modular repository contains only the 2 approved modules 
including their modular metadata...



"an enabled or default module"? Who enables a module in the buildroot? Is it
releng? The packager? Somebody else? Is it an explicit or implicit action?



"enabled" could happen as part of a mock config. That would be an
explicit action on the part of the person creating the mock config.
For the purposes of the discussion of Ursa Prime, you can ignore
"enabled". I was trying to be thorough and ended up being confusing
instead.


...and in Koji mock config, both are enabled and/or default?


Suppose we put maven:3.5 into the buildroot (I know we don't do that, but that
was the idea origianlly). How do I as a a maintainer of an ursine maven-foo
package BuildRequire the ursine maven package?



You don't.

>

Making a stream into the default stream is effectively making it
equivalent to a non-modular package for most purposes. It already
behaves this way for client installations; the content from a default
stream is the only version DNF "sees" if it happens to override a
non-modular package. On the client system, you'd need to `dnf module
disable maven` to be able to pull down the non-modular package.


I wonder what is the plan to handle this.

I see only two proper ways out of here:

1) We only allow default modular streams if they only contain packages that are 
retired from non-modular Fedora. Any request fro a default stream must be 
coordinated with all the relevant package maintainers and those packages must be 
retired from Fedora.


2) No default modular streams. (Yes, that again.)

In my opinion, if we allow the non-modular packages to stay when there is a 
default modular streams that "overrides" them, while effectively, our 
maintainers cannot use the non-modular packages in the buildroot and our users 
can only install the non-modular packages if they know how to disable a module, 
it creates a very confusing situation.



Ursa Prime is basically a Change to have the Koji build environment
behave the same way as the local environment would.

This is why we've been discussing the need for the artifacts of
default streams to behave *exactly* as if they were non-modular once
delivered. Because they effectively are. They were built differently
(though still in our infrastructure), but once built the requirement
is that they must then behave like the non-modular distro would have.


I hope we agree that they currently don't behave exactly like non-modular.


(Side-note: We retired the term "ursine" and replaced it with
"non-modular" because it was confusing for non-native English
speakers.)


Excellent choice. It was indeed rather confusing. Thank you.

Side-side-note: In my opinion, "Ursa Major/Prime" is equally confusing.


3. What is the local experience if the packager is using mock. What if they are
using rpmbuild directly?


For mock, if it's pointed at 

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Stephen Gallagher]

On Wed, Nov 13, 2019 at 1:24 PM Miro Hrončok  wrote:
>
> On 13. 11. 19 18:31, Stephen Gallagher wrote:
> > Sorry for the long overdue reply here. Answers to your questions are inline.
>
> Thank you.
>
> > On Wed, Oct 9, 2019 at 6:46 PM Miro Hrončok  wrote:
> > ...
> >> What I miss in the description is:
> >>
> >> 1. How does this thing actually work? is there an additional repository 
> >> composed
> >> from the default streams available in Koji only?
> >
> > An additional repository is composed from the default streams
> > (including the module metadata) and is merged with the standard
> > buildroot using Koji's new "bare" mode. This buildroot will be
> > available for non-Koji purposes (such as mock).
>
> Will this buildroot repository be mirrored or will it be more like the current
> "local" repository defined in mock configs?
>
> For reference, this is currently in the rawhide mock config:
>
> [local]
> name=local
> baseurl=https://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/
> cost=2000
> enabled=0
> skip_if_unavailable=False
>
> My concern is that using the "local" repository for mock builds is quite slow
> and IIRC puts Koji under unnecessary load.
>

The plan for the moment is to have it be the "local" repository. Given
that it will at least start out fairly small (and not need deltarpms),
we were going to see how well it held up. The longer-term plan will
probably be to mirror it.

> >> 2. How are conflicts between packages from the default streams and ursine
> >> package be handled?
> >
> > As long as the module metadata is present, an artifact with a given
> > name from an enabled or default module will effectively "hide" the
> > non-modular version.
>
> I have a hard time to understand this. Could you please dive into the details?
>
> "As long"? What does that mean? When they are not present and when they are 
> present?
>

Sorry, that was an English colloquialism. Basically I meant "unless we
screw up and don't sync the metadata to the repository". So as long as
a module stream providing an artifact with a given name is enabled or
default, non-modular RPMs of the same name will be invisible to the
solver.

> "an enabled or default module"? Who enables a module in the buildroot? Is it
> releng? The packager? Somebody else? Is it an explicit or implicit action?
>

"enabled" could happen as part of a mock config. That would be an
explicit action on the part of the person creating the mock config.
For the purposes of the discussion of Ursa Prime, you can ignore
"enabled". I was trying to be thorough and ended up being confusing
instead.

> Suppose we put maven:3.5 into the buildroot (I know we don't do that, but that
> was the idea origianlly). How do I as a a maintainer of an ursine maven-foo
> package BuildRequire the ursine maven package?
>

You don't.

Making a stream into the default stream is effectively making it
equivalent to a non-modular package for most purposes. It already
behaves this way for client installations; the content from a default
stream is the only version DNF "sees" if it happens to override a
non-modular package. On the client system, you'd need to `dnf module
disable maven` to be able to pull down the non-modular package.

Ursa Prime is basically a Change to have the Koji build environment
behave the same way as the local environment would.

This is why we've been discussing the need for the artifacts of
default streams to behave *exactly* as if they were non-modular once
delivered. Because they effectively are. They were built differently
(though still in our infrastructure), but once built the requirement
is that they must then behave like the non-modular distro would have.

(Side-note: We retired the term "ursine" and replaced it with
"non-modular" because it was confusing for non-native English
speakers.)


> > Unless you mean "conflicts" in some other sense, in which case I think
> > RPM-level conflicts should handle it.
>
> You understood "conflicts" exactly as I've meant it here, sorry for not being
> more explicit.
>

No problem, just wanted to make sure I was answering the correct question.

> >> 3. What is the local experience if the packager is using mock. What if 
> >> they are
> >> using rpmbuild directly?
> >
> > For mock, if it's pointed at the Koji-hosted buildroot repository, it
> > will work exactly as Koji does (with no local changes required).
>
> Makes sense, however, the information I miss is: Will it be pointed to the
> Koji-hosted buildroot repository by default or not? My point was that this
> should be part of the proposal before it was approved.
>

Yes, and you're right I will need to update the Change.

> > If
> > it's pointed at the public mirrors, it will need to have the Modular
> > repos enabled and it will have whatever is in the standard buildroot
> > plus default streams available there.
>
> Will it be pointed to the public mirrors with the Modular repos enabled?
> It previously was like that for a while and it was 

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Miro Hrončok]

On 13. 11. 19 19:21, Miro Hrončok wrote:
What you are describing here sounds like we are doing Ursa Major, renamed to 
Ursa Prime, because Ursa Prime was a no-go.


"because Ursa Major was a no-go"

Too many terms, sorry.


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Miro Hrončok]

On 13. 11. 19 18:31, Stephen Gallagher wrote:

Sorry for the long overdue reply here. Answers to your questions are inline.


Thank you.


On Wed, Oct 9, 2019 at 6:46 PM Miro Hrončok  wrote:
...

What I miss in the description is:

1. How does this thing actually work? is there an additional repository composed
from the default streams available in Koji only?


An additional repository is composed from the default streams
(including the module metadata) and is merged with the standard
buildroot using Koji's new "bare" mode. This buildroot will be
available for non-Koji purposes (such as mock).


Will this buildroot repository be mirrored or will it be more like the current 
"local" repository defined in mock configs?


For reference, this is currently in the rawhide mock config:

[local]
name=local
baseurl=https://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/
cost=2000
enabled=0
skip_if_unavailable=False

My concern is that using the "local" repository for mock builds is quite slow 
and IIRC puts Koji under unnecessary load.



2. How are conflicts between packages from the default streams and ursine
package be handled?


As long as the module metadata is present, an artifact with a given
name from an enabled or default module will effectively "hide" the
non-modular version.


I have a hard time to understand this. Could you please dive into the details?

"As long"? What does that mean? When they are not present and when they are 
present?

"an enabled or default module"? Who enables a module in the buildroot? Is it 
releng? The packager? Somebody else? Is it an explicit or implicit action?


Suppose we put maven:3.5 into the buildroot (I know we don't do that, but that 
was the idea origianlly). How do I as a a maintainer of an ursine maven-foo 
package BuildRequire the ursine maven package?



Unless you mean "conflicts" in some other sense, in which case I think
RPM-level conflicts should handle it.


You understood "conflicts" exactly as I've meant it here, sorry for not being 
more explicit.



3. What is the local experience if the packager is using mock. What if they are
using rpmbuild directly?


For mock, if it's pointed at the Koji-hosted buildroot repository, it
will work exactly as Koji does (with no local changes required).


Makes sense, however, the information I miss is: Will it be pointed to the 
Koji-hosted buildroot repository by default or not? My point was that this 
should be part of the proposal before it was approved.



If
it's pointed at the public mirrors, it will need to have the Modular
repos enabled and it will have whatever is in the standard buildroot
plus default streams available there.


Will it be pointed to the public mirrors with the Modular repos enabled?
It previously was like that for a while and it was reverted as a wrong behavior.


One purpose of this Change is to
reconcile those two cases so they are the same.


If the cases are the same, isn't it actually what was originally proposed as 
Ursa Major? I am seriously confused here, because I understood this change as it 
is explicitly doing a different thing, a nonmodular repository composed of the 
modular content. So that our downstreams can consume it "traditionally" without 
adapting their tooling.


What you are describing here sounds like we are doing Ursa Major, renamed to 
Ursa Prime, because Ursa Prime was a no-go. Because I don't assume ill intent, I 
am absolutely sure that there is something I'm not getting here, but it it very 
hard to figure out what.


Could you please, in great detail describe the differences between the two and 
how this approved change addresses all the reasons that the previous attempt was 
forbidden?



Right now, the Koji
buildroot will be slightly different from the one mock would see
because FESCo asked us to only allow two modules in the Koji buildroot
for testing.


Why would mock "see" different things based on restrictions FESCo asked to do?
My primary concern is that by default, mock sees what Koji sees.

By this change and by what was approved by FESCo (to clarify: I voted against 
this), we are adding new things to what Koji sees. So I am really not sure what 
actually happens to what mock sees:


1. Do we add only the 2 FESCo approved modules to what mock sees?
2. Do we add all default modular streams to what mock sees?
3. Do we keep mock to only see ursine content?

My opinion:

1. is ideal, but was not approved nor documented in the change proposal
2. is not ideal and was not approved nor documented in the change proposal
3. is not ideal, but is status quo, hence requires no approval/change


Prior to this Change, the Koji buildroot would have only
the non-modular content.


Correct.


4. How are we handling default streams with dependencies on non-default streams
of other modules?


Throughout these discussions, it has been made clear that the best
option for now will be to disallow default streams from depending on
non-default streams in Fedora. 

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Stephen Gallagher]

Sorry for the long overdue reply here. Answers to your questions are inline.

On Wed, Oct 9, 2019 at 6:46 PM Miro Hrončok  wrote:
...
> What I miss in the description is:
>
> 1. How does this thing actually work? is there an additional repository 
> composed
> from the default streams available in Koji only?

An additional repository is composed from the default streams
(including the module metadata) and is merged with the standard
buildroot using Koji's new "bare" mode. This buildroot will be
available for non-Koji purposes (such as mock).

> 2. How are conflicts between packages from the default streams and ursine
> package be handled?

As long as the module metadata is present, an artifact with a given
name from an enabled or default module will effectively "hide" the
non-modular version.

Unless you mean "conflicts" in some other sense, in which case I think
RPM-level conflicts should handle it.


> 3. What is the local experience if the packager is using mock. What if they 
> are
> using rpmbuild directly?

For mock, if it's pointed at the Koji-hosted buildroot repository, it
will work exactly as Koji does (with no local changes required). If
it's pointed at the public mirrors, it will need to have the Modular
repos enabled and it will have whatever is in the standard buildroot
plus default streams available there. One purpose of this Change is to
reconcile those two cases so they are the same. Right now, the Koji
buildroot will be slightly different from the one mock would see
because FESCo asked us to only allow two modules in the Koji buildroot
for testing. Prior to this Change, the Koji buildroot would have only
the non-modular content.

> 4. How are we handling default streams with dependencies on non-default 
> streams
> of other modules?

Throughout these discussions, it has been made clear that the best
option for now will be to disallow default streams from depending on
non-default streams in Fedora. This *can* (but may not) change in the
future once we have worked out the explicit-vs-implicit enablement and
upgrade questions. But for now, it seems reasonable to put that policy
constraint on it.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Zbigniew Jędrzejewski-Szmek]

On Fri, Oct 18, 2019 at 02:00:11PM -0700, Adam Williamson wrote:
> On Fri, 2019-10-18 at 16:25 -0400, Robbie Harwood wrote:
> > Christopher Engelhard  writes:
> > 
> > > On 18.10.19 17:21, Robbie Harwood wrote:
> > > 
> > > > While you're right that the solutions from source distros (i.e., NixOS
> > > > and Gentoo) would be very hard to adapt, binary distros have also solved
> > > > this problem in different ways.  I'm most familiar with Debian's
> > > > solution (virtual packages[2], provides:, and alternatives [1]) which
> > > > to my mind maps much more clearly onto Fedora's setup.  Obviously we
> > > > can't use their code wholesale without migrating to APT, but as you say,
> > > > the goal is to derive inspiration.
> > > > 
> > > > 1: https://wiki.debian.org/DebianAlternatives
> > > > 2: 
> > > > https://www.debian.org/doc/debian-policy/ch-relationships.html#s-virtual
> > > 
> > > I'm not a Fedora packager (yet[1]), so correct me if I'm
> > > misunderstanding things completely, but is there any reason not to
> > > adapt the existing RPM provides: functionality for this?
> > 
> > I'm not aware of technical reasons not to do this - as Neal elaborated
> > on, this is already possible.
> 
> I don't think there is one. But it's not a perfect solution either.
 
> Modularity is intended to do that - to provide a framework and some
> technical backing for treating alternate groups of packages that
> provide different implementations of the same thing *as* groups,
> essentially.
> 
> I mean, it's not like the Modularity team wasn't aware of these other
> approaches, I'm pretty sure they were discussed. It might be reasonable
> to go back and check the various docs and blog posts and things the
> modularity team put out before just suggesting 'hey what about this
> other thing', especially where 'this other thing' is something you
> could reasonably imagine at least one of them would probably have heard
> or thought of...

Well, you might think that, but it's not clear that this was the case.
Let's not assume one way or the other.

> I mean, we're talking about *stacks* here, remember. FreeIPA isn't one
> package - you can't just have 'freeipa4' and 'freeipa5' both providing
> 'freeipa' and call it a day. There's a whole ton of stuff that goes in
> there. 'freeipa5' might need different version of ten different other
> components compared to 'freeipa4'.
> 
> So, you're no longer looking at one pair of packages with the same
> 'provides', you're looking at a dozen, and someone needs to be keeping
> track of which ones go together and what they're all for in the first
> place. But with this approach you're not building any infrastructure
> for *doing* that.

I would really like this explained. Taking this particular example, to
let dnf install all dependencies properly, freeipa5 must already list
*all* dependencies through rpm Requires/Conflicts/Recommends/Suggests,
including versions. Likewise, freeipa4 must also do that. It doesn't
matter if the rpms are built normally or in a module, listing all deps
is needed in both cases. If the rpm doesn't work with all versions of
dependencies, it might be fine during initial installation, where you
get everything latest by default, but will break if the user does a
partial upgrade.

This means that in one case the user says 'dnf install freeipa5' and they
get the full stack, and in the other case they say 'dnf install freeipa4'
and they get a different (partially overlapping) set of recursive deps.

> These things are clearly a conceptual lump, but
> you're not really providing a way to express or work with that.

Why not? What is wrong with having rpm Requires?

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Lukas Ruzicka]

3) We need to get the policy I described above written onto -stone
> tablets- the Packaging Guidelines and then we need to go and make any
> stream that isn't compliant with it a non-default stream.
>

Thank you. If we want to use default streams, then we indeed need a strict
policy on how they are formed and we need to make sure, the
requirements are met here. I dare to say, that we should block on every
default stream that would not meet the requirements.

-- 

Lukáš Růžička

FEDORA QE, RHCE

Red Hat



Purkyňova 115

612 45 Brno - Královo Pole

lruzi...@redhat.com
TRIED AND PERSONALLY TESTED, ERGO TRUSTED. 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Adam Williamson]

On Fri, 2019-10-18 at 16:25 -0400, Robbie Harwood wrote:
> Christopher Engelhard  writes:
> 
> > On 18.10.19 17:21, Robbie Harwood wrote:
> > 
> > > While you're right that the solutions from source distros (i.e., NixOS
> > > and Gentoo) would be very hard to adapt, binary distros have also solved
> > > this problem in different ways.  I'm most familiar with Debian's
> > > solution (virtual packages[2], provides:, and alternatives [1]) which
> > > to my mind maps much more clearly onto Fedora's setup.  Obviously we
> > > can't use their code wholesale without migrating to APT, but as you say,
> > > the goal is to derive inspiration.
> > > 
> > > 1: https://wiki.debian.org/DebianAlternatives
> > > 2: 
> > > https://www.debian.org/doc/debian-policy/ch-relationships.html#s-virtual
> > 
> > I'm not a Fedora packager (yet[1]), so correct me if I'm
> > misunderstanding things completely, but is there any reason not to
> > adapt the existing RPM provides: functionality for this?
> 
> I'm not aware of technical reasons not to do this - as Neal elaborated
> on, this is already possible.

I don't think there is one. But it's not a perfect solution either.

I mean, we're talking about *stacks* here, remember. FreeIPA isn't one
package - you can't just have 'freeipa4' and 'freeipa5' both providing
'freeipa' and call it a day. There's a whole ton of stuff that goes in
there. 'freeipa5' might need different version of ten different other
components compared to 'freeipa4'.

So, you're no longer looking at one pair of packages with the same
'provides', you're looking at a dozen, and someone needs to be keeping
track of which ones go together and what they're all for in the first
place. But with this approach you're not building any infrastructure
for *doing* that. These things are clearly a conceptual lump, but
you're not really providing a way to express or work with that.

Modularity is intended to do that - to provide a framework and some
technical backing for treating alternate groups of packages that
provide different implementations of the same thing *as* groups,
essentially.

I mean, it's not like the Modularity team wasn't aware of these other
approaches, I'm pretty sure they were discussed. It might be reasonable
to go back and check the various docs and blog posts and things the
modularity team put out before just suggesting 'hey what about this
other thing', especially where 'this other thing' is something you
could reasonably imagine at least one of them would probably have heard
or thought of...
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Robbie Harwood]

Christopher Engelhard  writes:

> On 18.10.19 17:21, Robbie Harwood wrote:
>
>> While you're right that the solutions from source distros (i.e., NixOS
>> and Gentoo) would be very hard to adapt, binary distros have also solved
>> this problem in different ways.  I'm most familiar with Debian's
>> solution (virtual packages[2], provides:, and alternatives [1]) which
>> to my mind maps much more clearly onto Fedora's setup.  Obviously we
>> can't use their code wholesale without migrating to APT, but as you say,
>> the goal is to derive inspiration.
>> 
>> 1: https://wiki.debian.org/DebianAlternatives
>> 2: https://www.debian.org/doc/debian-policy/ch-relationships.html#s-virtual
>
> I'm not a Fedora packager (yet[1]), so correct me if I'm
> misunderstanding things completely, but is there any reason not to
> adapt the existing RPM provides: functionality for this?

I'm not aware of technical reasons not to do this - as Neal elaborated
on, this is already possible.

> That is essentially what Archlinux does with their pacman, where an
> second version of the same program will typically both Provide: and
> Conflict: with the 'default' package. It's extensively used by users to
> provide e.g. development versions of stable packages, see e.g.
> inkscape-git [2].
>
> - if you install inkscape, you get the default version
> - if you install inkscape-git, you get the development version
> - when resolving dependencies, pacman knows that inkscape-git provides
> the capabilities of (Provides:) but cannot be installed together with
> (Conflicts:) of the package named inkscape
> - otherwise, they are two separate packages, so you don't have the
> problem of implicitly switching streams
>
> Parallel installability is taken care of with compat packages as usual,
> though if the packages happen to be parallel installable, there's
> probably no reason not to allow that via 'provides:' without 'conflicts:'
>
> What I like about this approach that all this magic only comes into play
> when resolving dependencies. In all other circumstances they are
> packages like any other, and are treated as such.

Yup!

Thanks,
--Robbie


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Christopher Engelhard]

On 18.10.19 17:21, Robbie Harwood wrote:
> While you're right that the solutions from source distros (i.e., NixOS
> and Gentoo) would be very hard to adapt, binary distros have also solved
> this problem in different ways.  I'm most familiar with Debian's
> solution (virtual packages[2], provides:, and alternatives [1]) which
> to my mind maps much more clearly onto Fedora's setup.  Obviously we
> can't use their code wholesale without migrating to APT, but as you say,
> the goal is to derive inspiration.
> 
> Thanks,
> --Robbie
> 
> 1: https://wiki.debian.org/DebianAlternatives
> 2: https://www.debian.org/doc/debian-policy/ch-relationships.html#s-virtual

I'm not a Fedora packager (yet[1]), so correct me if I'm
misunderstanding things completely, but is there any reason not to adapt
the existing RPM provides: functionality for this?

That is essentially what Archlinux does with their pacman, where an
second version of the same program will typically both Provide: and
Conflict: with the 'default' package. It's extensively used by users to
provide e.g. development versions of stable packages, see e.g.
inkscape-git [2].

- if you install inkscape, you get the default version
- if you install inkscape-git, you get the development version
- when resolving dependencies, pacman knows that inkscape-git provides
the capabilities of (Provides:) but cannot be installed together with
(Conflicts:) of the package named inkscape
- otherwise, they are two separate packages, so you don't have the
problem of implicitly switching streams

Parallel installability is taken care of with compat packages as usual,
though if the packages happen to be parallel installable, there's
probably no reason not to allow that via 'provides:' without 'conflicts:'

What I like about this approach that all this magic only comes into play
when resolving dependencies. In all other circumstances they are
packages like any other, and are treated as such.


Christopher

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1756582
[2] https://aur.archlinux.org/packages/inkscape-git
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Neal Gompa]

On Fri, Oct 18, 2019 at 11:43 AM Randy Barlow
 wrote:
>
> On Fri, 2019-10-18 at 11:21 -0400, Robbie Harwood wrote:
> > Obviously we
> > can't use their code wholesale without migrating to APT, but as you
> > say,
> > the goal is to derive inspiration.
>
> But yeah as you say here, my original point was more that we could
> learn from what others did and integrate similar concepts into RPM/dnf,
> especially since what they did was simpler, and as a result of that
> simplicity didn't struggle to gain adoption. Those concepts were also
> easier to write the code to make them happen. For example, adding a
> slot field to the RPM would be pretty easy and I think is obvious that
> we wouldn't need to create build services like Ursa Major to make that
> work. We would need a change to Koji and dnf to consider the slot as
> part of a package's uniqueness, but I think that's a small change in
> comparison. It's easier to create and easier to use, which I see as a
> win-win.
>
> Thanks for mentioning the Debian solution. I wasn't familiar with that
> one. This means we have even more solutions to study for inspiration.
>
> Too fast, too slow is a solved problem and there are existing solutions
> to learn from or even adapt. We don't need to make our own.

RPM + DNF actually already supports Debian's solution.

Philosophically, RH/Fedora systems don't typically use alternatives as
aggressively as Mageia, SUSE, or Debian systems. We do actually
generate virtual names and use them, and we can use them to satisfy
things like this. We already do this with Rust by having parallel
installable stuff with common names that work with ranged
dependencies.

There are other things we could do along these lines, too...



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Randy Barlow]

On Fri, 2019-10-18 at 11:21 -0400, Robbie Harwood wrote:
> Obviously we
> can't use their code wholesale without migrating to APT, but as you
> say,
> the goal is to derive inspiration.

I honestly think it should be on the table to consider switching to a
different packaging technology than rpm/dnf. Collaborating with other
distributions to solve common problems has benefits. We do that to some
extent with SUSE (which is cool!), but not to the fullest extent one
could imagine.

But yeah as you say here, my original point was more that we could
learn from what others did and integrate similar concepts into RPM/dnf,
especially since what they did was simpler, and as a result of that
simplicity didn't struggle to gain adoption. Those concepts were also
easier to write the code to make them happen. For example, adding a
slot field to the RPM would be pretty easy and I think is obvious that
we wouldn't need to create build services like Ursa Major to make that
work. We would need a change to Koji and dnf to consider the slot as
part of a package's uniqueness, but I think that's a small change in
comparison. It's easier to create and easier to use, which I see as a
win-win.

Thanks for mentioning the Debian solution. I wasn't familiar with that
one. This means we have even more solutions to study for inspiration.

Too fast, too slow is a solved problem and there are existing solutions
to learn from or even adapt. We don't need to make our own.


signature.asc
Description: This is a digitally signed message part
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Robbie Harwood]

Stephen John Smoogen  writes:

> On Thu, 17 Oct 2019 at 14:15, Randy Barlow  
> wrote:
>
>> Or better, can we employ a solution that another distribution has
>> developed?
>
> Not without using their packaging system, their build system and their
> other design choices. Working out slots would mean needing to make
> changes into how RPM works and how yum/dnf work. It might also not be
> possible because a bit of Gentoo's magic is letting the local system
> build all the different slot choices instead of having to build all
> the combinatorics that having 3 different glibc and N gcc compilers
> would need. To do the magic NixOS does.. we need to eject the FHS and
> use a similar system. At that point, we aren't developing Fedora
> anymore.. we are developing a clone of NixOS or Gentoo. [And there
> would be no magic way to move from a Fedora 33 system to Fedora-Nix-34
> or Fedora-Gen-34.. at which point we might as well just call the whole
> thing from scratch.] If we are going that far we might as well rewrite
> conary in python3 or rust and start from there...
>
> So any solution will have to 'learn' the lessons of these groups but
> design and write a solution from scratch to meet them.

While you're right that the solutions from source distros (i.e., NixOS
and Gentoo) would be very hard to adapt, binary distros have also solved
this problem in different ways.  I'm most familiar with Debian's
solution (virtual packages[2], provides:, and alternatives [1]) which
to my mind maps much more clearly onto Fedora's setup.  Obviously we
can't use their code wholesale without migrating to APT, but as you say,
the goal is to derive inspiration.

Thanks,
--Robbie

1: https://wiki.debian.org/DebianAlternatives
2: https://www.debian.org/doc/debian-policy/ch-relationships.html#s-virtual


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Nico Kadel-Garcia]

On Thu, Oct 17, 2019 at 8:09 AM Stephen Gallagher  wrote:
>
> On Thu, Oct 17, 2019 at 7:53 AM Miro Hrončok  wrote:
> > > That was a representative example. I came up with it at 11pm after a
> > > long day. Don't read too much into the specifics. The point was that
> > > builds may require newer or older software than we have available in
> > > the non-modular buildroot, but don't require them at runtime.
> >
> > Yes, they do. There are many ways to solve the problem, from compat packages
> > trough various tag-magic to buildroot overrides in custom side tags. We 
> > have had
> > this problem for years and we were able to deal with it quite well.
> >
>
> Please let's not confuse "we have figured out how to work around it by
> making our packagers do a lot of tricky hacks" with "able to deal with
> it quite well". This is a major issue for non-expert packagers. I
> don't have statistics on how many people give up on a package because
> of this issue, but I know *I* have done so (and I'd consider myself
> more experienced than most people at packaging).
>
> One of the (often un- or misinformed) major arguments people keep
> using against Modularity is "it makes packaging harder!". This is one
> place where it makes things *much* easier on the packagers. It's a
> clear reduction in complexity.

As someone who's published over 500 RPMs to various commercial and
free software environments, it's a complete waste of my time, it's
breaking my build my build environments and wasting time I'd rather
spend on building or updating individual components without confusion
about maintaining multiple dependency trees.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Randy Barlow]

On Thu, 2019-10-17 at 15:04 -0400, Stephen John Smoogen wrote:
> Not without using their packaging system, their build system and
> their
> other design choices.

Frankly, this is not a bad caveat. Keep in mind that we also had to
change our build system for modularity.

>  Working out slots would mean needing to make
> changes into how RPM works and how yum/dnf work.

This is also true for modularity.

>  It might also not be
> possible because a bit of Gentoo's magic is letting the local system
> build all the different slot choices instead of having to build all
> the combinatorics that having 3 different glibc and N gcc compilers
> would need.

Conary had use flags without the local system doing the builds. But in
general, I think the slots thing would not require any local builds -
it's just a way to specify cleanly which versions of something can be
available for parallel install and which cannot.

By the way, I think I started using Gentoo in 2004 or so, and back then
they didn't have parallel installability, but they did have parallel
availability, even without slots. They just had various versions of
each package available in the same repository at the same time, and I
could use the package manager to express which one I want.

>  To do the magic NixOS does.. we need to eject the FHS and
> use a similar system. At that point, we aren't developing Fedora
> anymore.. we are developing a clone of NixOS or Gentoo.

Well there's more to the distro than the package manager. I think we
could use great tools that other distros have made instead of having to
make our own just because we want to be more separate from them.

>  [And there
> would be no magic way to move from a Fedora 33 system to Fedora-Nix-
> 34
> or Fedora-Gen-34.. at which point we might as well just call the
> whole
> thing from scratch.] If we are going that far we might as well
> rewrite
> conary in python3 or rust and start from there...

I agree that upgrading would be hard.

Resurrecting conary for Python 3 would be awesome, I'm sad that it is
just sitting there in Python 2.old not being used - it's really good.

Rust is also great.


signature.asc
Description: This is a digitally signed message part
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Orion Poplawski]

On 10/17/19 1:32 PM, Matthew Miller wrote:
> On Wed, Oct 16, 2019 at 03:05:43PM -0700, John M. Harris Jr wrote:
>> Realistically, I believe that default streams themselves are something we
>> should avoid, if the cost is low, and it is. There are many users,
>> probably the vast majority of users, that don't use Modularity. It's great
>> to have the option available, but to force it upon them is really
>> unfortunate.
> 
> I don't think characterizing this as "forced upon users" is a good framing.
> It's unnecessarily hostile. Users who happen to get packages which are in a
> default stream shouldn't experience any practical difference from having a
> non-modular package.
> 

I don't like the language either, but it's undeniable that dealing with
modules is different than dealing with normal packages.  Enabling a module
excludes those packages in the module from "normal" depsolving.  I've just run
into that with the RHEL8 rhn-tools module which provides koan (and excludes
cobbler).  I couldn't install my own cobbler or koan packages (with higher
EVRs) from my copr until I disabled that module - and it was not easy to
figure that out.

I'm also not adverse to learning new ways of doing things.  I think modules
can still provide some clear benefits to users and developers eventually, but
let's not pretend there are no differences involved.

-- 
Orion Poplawski
Manager of NWRA Technical Systems  720-772-5637
NWRA, Boulder/CoRA Office FAX: 303-415-9702
3380 Mitchell Lane   or...@nwra.com
Boulder, CO 80301 https://www.nwra.com/



smime.p7s
Description: S/MIME Cryptographic Signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Fabio Valentini]

On Thu, Oct 17, 2019 at 9:33 PM Matthew Miller  wrote:
>
> On Wed, Oct 16, 2019 at 03:05:43PM -0700, John M. Harris Jr wrote:
> > Realistically, I believe that default streams themselves are something we
> > should avoid, if the cost is low, and it is. There are many users,
> > probably the vast majority of users, that don't use Modularity. It's great
> > to have the option available, but to force it upon them is really
> > unfortunate.
>
> I don't think characterizing this as "forced upon users" is a good framing.
> It's unnecessarily hostile.

I agree that the language is a tad aggressive, but still:

> Users who happen to get packages which are in a
> default stream shouldn't experience any practical difference from having a
> non-modular package.

And yet, the differences *are undeniably there*, and users are
expected to fix issues this causes on their systems down the road
themselves, even if they didn't consciously "opted in" to anything.
I'd not characterize this as "forcing something upon users", but it's
still getting in through the back door while they're possibly not
looking.

(And that's why the first thing I do for all my new fedora installs is
to disable modular repos, just so no modules get installed
"accidentally".)

Fabio

> --
> Matthew Miller
> 
> Fedora Project Leader
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Matthew Miller]

On Wed, Oct 16, 2019 at 03:05:43PM -0700, John M. Harris Jr wrote:
> Realistically, I believe that default streams themselves are something we
> should avoid, if the cost is low, and it is. There are many users,
> probably the vast majority of users, that don't use Modularity. It's great
> to have the option available, but to force it upon them is really
> unfortunate.

I don't think characterizing this as "forced upon users" is a good framing.
It's unnecessarily hostile. Users who happen to get packages which are in a
default stream shouldn't experience any practical difference from having a
non-modular package.

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Stephen John Smoogen]

On Thu, 17 Oct 2019 at 14:15, Randy Barlow  wrote:
>

> Could we think of a solution that is simple so that packagers can more
> easily understand how it works?

The issue is how many different choices are you allowing and where you
are allowing them to be made. A lot of the gentoo and nixos seem to be
made on the personal system. I choose how I want to build out my box
and it builds itself to do that. If you push the decisions up into the
central build system, then you need to start working with other people
and letting them mess with your decisions. That means various design
choices no longer are simple.. and are less technical. They are
instead policy and bureaucracy in order to grease the interactions and
different opinions to get a compromise or in the end a hard decision
even when no one wants it.

The problem I found with both the documentation and most of our
conversations is that we have been trying to treat this as a
technology problem when it is a social one. A distribution is a social
contract between different packagers to make a 'product' (I am trying
for a word which is less commercial so if you know one.. use it
instead). A module is just a smaller social contract where we need to
make a set of things work together in a way that works both with the
larger product and its own self. Most of the rules in writing a good
spec file are meant to make it easier that someone else can maintain
it later.. that is a social requirement. We might dress it up with
'MUST have %{0fedora}' but it is a social contract. The rpm doesn't
care if I use %0fedora or 1. [Man I have digressed into finding the
source of the Nile somewhere.. ok pull it together smooge]

In the end, we need to work out what the social wants/needs for the
technological marvel we create are before we build it. The larger the
number of people you are working with said marvel, the more important
that social rules/upkeep is. Putting the brakes on the car after it is
going 100mph is a little late.

> Or better, can we employ a solution
> that another distribution has developed?

Not without using their packaging system, their build system and their
other design choices. Working out slots would mean needing to make
changes into how RPM works and how yum/dnf work. It might also not be
possible because a bit of Gentoo's magic is letting the local system
build all the different slot choices instead of having to build all
the combinatorics that having 3 different glibc and N gcc compilers
would need. To do the magic NixOS does.. we need to eject the FHS and
use a similar system. At that point, we aren't developing Fedora
anymore.. we are developing a clone of NixOS or Gentoo. [And there
would be no magic way to move from a Fedora 33 system to Fedora-Nix-34
or Fedora-Gen-34.. at which point we might as well just call the whole
thing from scratch.] If we are going that far we might as well rewrite
conary in python3 or rust and start from there...

So any solution will have to 'learn' the lessons of these groups but
design and write a solution from scratch to meet them.

-- 
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Randy Barlow]

On Thu, 2019-10-17 at 08:08 -0400, Stephen Gallagher wrote:
> One of the (often un- or misinformed) major arguments people keep
> using against Modularity is "it makes packaging harder!".

One thing I've found to be a problem with modularity is that it's easy
to be un- or misinformed. I spent a lot of time reading the docs (and
I'm glad there are docs, I'm not criticizing the docs) and I still
found it difficult to modularize my rpick package.

My experiences as a packager for the other distributions I've worked
with that offer parallel installability (which necessarily means
parallel availability) was that I didn't need to know very much to do
it. It was generally obvious what to do to make my package use slots,
for example.

I think the problem is that modularity is complicated enough that it's
hard to know it well without investment.

Could we think of a solution that is simple so that packagers can more
easily understand how it works? Or better, can we employ a solution
that another distribution has developed?


signature.asc
Description: This is a digitally signed message part
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Alexander Scheel]

- Original Message -
> From: "Randy Barlow" 
> To: "Development discussions related to Fedora" 
> 
> Sent: Thursday, October 17, 2019 1:18:08 PM
> Subject: Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular 
> Buildroot
> 
> On Thu, 2019-10-17 at 12:56 -0400, Randy Barlow wrote:
> > I
> > had to write a yaml file that listed hashes of every dependency of
> > rpick, and every dependency of those dependencies, and their
> > dependencies, and so on.
> 
> By the way, I didn't actually end up doing this, Igor did it for me. I
> didn't mean to take credit, I mean more that "I, the packager, had to
> do this", not "I, Randy, had to do this". Igor rocks.

Just for the obvious reply... :)


`ref` behaves just like `git checkout` would. So yeah, you can pass a very
explicit hash and then you won't get any updates until you bump the hash
in the spec again. However, you could pass tags (like you mention in the
comment next to each hash) or branches instead. The latter is what some
other modules do, like the eclipse one:

https://src.fedoraproject.org/modules/eclipse/blob/2019-06/f/eclipse.yaml#_1069

See also:

https://github.com/fedora-modularity/libmodulemd/blob/master/spec.v2.yaml#L268-L273


In the latter case (in RHEL at least), you have to commit a (potentially
empty) commit to the module and then a rebuild of the module will pull in
the latest contents from the branches you specify for each component RPM.


Perhaps we need to collect a "tips and tricks" section for modularity?


HTH, 

Alex


> 
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Randy Barlow]

On Thu, 2019-10-17 at 12:56 -0400, Randy Barlow wrote:
> I
> had to write a yaml file that listed hashes of every dependency of
> rpick, and every dependency of those dependencies, and their
> dependencies, and so on.

By the way, I didn't actually end up doing this, Igor did it for me. I
didn't mean to take credit, I mean more that "I, the packager, had to
do this", not "I, Randy, had to do this". Igor rocks.


signature.asc
Description: This is a digitally signed message part
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Randy Barlow]

On Thu, 2019-10-17 at 10:52 -0400, Randy Barlow wrote:
> I've always liked Gentoo's solution to "too fast too slow"
> with their slots mechanism.

I realized it would be good if I explained what this is in more detail
for those who aren't familiar.

The slot is another field on the package, and the package manager
ensures that you only have one installed from each slot of a package.
That's basically it.

Conventionally, the slot format is often a leading portion of the
package's version that users might want to be able to install side by
side. Using Python as an example, Gentoo usually would use things like
"2.7" or "3.6" in the "slot" field, i.e., leaving out the .z component
of the version. This means users could install 2.7, 3.5, 3.6, and 3.7
at the same time, but that 3.7.0 and 3.7.1 would not be offered. Which
granularity of slots are offered depend on the package and the packager
(i.e., the packager makes sure the package can be simultaneously
installed at the granularity of the slot, and of course also volunteers
to maintain the various versions.)

I like this as a user because it's presented in a simple way to me.
Slotted packages don't feel any different than non-slotted packages. I
use the same commands to install them as I do any other package, unless
I want a specific version, in which case I just add some <>= type
statement on my install command to specify which version I want
(otherwise I get the default, which is typically latest available as
long as it doesn't conflict with other packages I have installed.)

I also like this as a packager, because my packaging experience is also
pretty much the same as a non-slotted package, with the obvious caveat
that I also need to make sure that the slots can co-exist. But what I
mean is that the tooling and packaging file format is pretty much the
same. No new things to learn, no other services. It's just another
package.


signature.asc
Description: This is a digitally signed message part
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Randy Barlow]

On Thu, 2019-10-17 at 08:08 -0400, Stephen Gallagher wrote:
> One of the (often un- or misinformed) major arguments people keep
> using against Modularity is "it makes packaging harder!". This is one
> place where it makes things *much* easier on the packagers. It's a
> clear reduction in complexity.

I tried to maintain a package I have, rpick, as a module, and I found
it to be difficult. In order to package the module version of rpick, I
had to write a yaml file that listed hashes of every dependency of
rpick, and every dependency of those dependencies, and their
dependencies, and so on. This file:

https://src.fedoraproject.org/modules/rpick/blob/latest/f/rpick.yaml

I also maintain another Rust package, yubibomb, which is just a normal
spec file:

https://src.fedoraproject.org/rpms/rust-yubibomb/blob/master/f/rust-yubibomb.spec

I've heard it claimed that there is something specific to Rust here,
but to be honest the explanation of that was over my head. I have also
heard it claimed that Modularity was specifically supposed to help with
Rust packaging, but I haven't personally felt like it was helpful due
to the yaml file.

I'm sure my information is out of touch, but I'm sharing my experience
as someone who doesn't have a lot of time to go learn something new -
my reaction is that it's easier to maintain the "traditional" version.

My other reaction is that other distributions have solved the "too
fast, too slow" problem in simpler ways, as I wrote about elsewhere on
this thread, and I am curious why we didn't adopt their solutions (or
at least use them to guide ourselves). I've not packaged for Nix
before, but I have packaged for rPath and for Gentoo and I found their
solutions to be easy and natural.


signature.asc
Description: This is a digitally signed message part
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Randy Barlow]

On Thu, 2019-10-17 at 03:53 +0200, Kevin Kofler wrote:
> The user-friendly approach for that is to use a parallel-installable 
> compatibility package (with a suffixed Name, such as django1.6)
> instead of a 
> module.

I've always liked Gentoo's solution to "too fast too slow" (which has
been around for what seems like more than 10 years, though I'm not sure
the exact date they introduced it, so let's just say "a long time")
with their slots mechanism. Their packages just have another metadata
field that is called slot, and typically corresponds to a portion of
the version that indicates how granular the packager wants to go with
parallel availability (and in what seems like all cases I've noticed,
they also have parallel installability, though I'm not confident enough
to claim they offer that for all packages without verification). For
example, I have Python 2.7 and 3.6 one one of my machines, but it's not
called python2 and python3, both are just called "python":

$ equery l python
dev-lang/python-2.7.15
dev-lang/python-3.6.5

And all Python packages I install don't need python2- or python3-
prefixes, they can just use their normal names and Gentoo does the
right thing based on which Python versions I have installed. It
actually doesn't even install n versions of each Python package, just
one that determines which Pythons you have and just gets installed for
those versions. It's great. rPath's Conary also had some of these
concepts (and had my favorite packaging format that I've used - your
"spec file" is just a Python class. No DSL, and you can take advantage
of inheritance to make pretty simple and clean spec files!).

Why not just stand on the shoulders of giants and add a "slot" field to
the spec file? Then you can have everything be regular RPMs. No
complicated build services to figure out. It's simpler.

I've also wondered why we didn't take a queue from NixOS, which seems
to have also solved these problems a long time ago in an even more
advanced way than Gentoo did. It'd be a bigger change than using slots,
but I think it'd be even more advantageous.

Are we re-implementing something that's been solved by others before
instead of joining them upstream? We aren't solving the "too fast, too
slow" problem no matter what we do, because it was already solved long
ago by others. Why not just benefit from their efforts and adopt their
technology and experiences to Fedora, rather than doing it ourselves?
Isn't that the advantage of open source software?


signature.asc
Description: This is a digitally signed message part
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Neal Gompa]

On Thu, Oct 17, 2019 at 8:37 AM Stephen Gallagher  wrote:
>
> On Thu, Oct 17, 2019 at 8:33 AM Neal Gompa  wrote:
> > People regularly look at NEVRAs to identify whether there are
> > "broken"/"old" packages to clean up, and when you see .fc29 installed
> > on an otherwise .fc31 system, it looks like something has gone wrong.
> > If packages are intended to be built once and pushed to multiple
> > releases, it shouldn't have the DistTag suffix.
>
> OK, I misunderstood you because DistTag and %{?dist} are actually
> distinct things in RPM and you're using the wrong one here :)
>

It's called that in the Packaging Guidelines:
https://docs.fedoraproject.org/en-US/packaging-guidelines/DistTag/

It should probably be named differently now that there's an _actual_ DistTag.

> That's an interesting thought and we should consider what to do about
> it. Mind opening a ticket on https://pagure.io/modularity so we don't
> forget?

Done: https://pagure.io/modularity/issue/157




--
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Stephen Gallagher]

On Thu, Oct 17, 2019 at 8:33 AM Neal Gompa  wrote:
> People regularly look at NEVRAs to identify whether there are
> "broken"/"old" packages to clean up, and when you see .fc29 installed
> on an otherwise .fc31 system, it looks like something has gone wrong.
> If packages are intended to be built once and pushed to multiple
> releases, it shouldn't have the DistTag suffix.

OK, I misunderstood you because DistTag and %{?dist} are actually
distinct things in RPM and you're using the wrong one here :)

That's an interesting thought and we should consider what to do about
it. Mind opening a ticket on https://pagure.io/modularity so we don't
forget?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Neal Gompa]

On Thu, Oct 17, 2019 at 7:53 AM Stephen Gallagher  wrote:
>
> On Thu, Oct 17, 2019 at 7:48 AM Neal Gompa  wrote:
> >
> > On Thu, Oct 17, 2019 at 7:42 AM Stephen Gallagher  
> > wrote:
> > >
> > > Similarly, the example of "build on Rawhide, run anywhere" was
> > > backwards. I should have said "build on oldest supported Fedora, carry
> > > through".
> >
> > Modules currently fail at this because they have a platform
> > dependency.
>
> I meant to address that in an earlier email when I saw you mention
> that. It was wrong then, too.
>
> Most modules *specify* a platform dependency, but they *do not have to*.
>
> Take a look at the maven module, for example:
> https://src.fedoraproject.org/modules/maven/blob/3.5/f/maven.yaml
>
> You will notice that it specifies F29 as a build requirement but
> specifies '[ ]' for the platform: this means "Build it on F29, run it
> anywhere".
>

Ah, so we already have the BR/R split for module deps.

> > And we could *easily* permit it for non-modular packages
> > if we actually wanted it. I think we've generally said we don't want
> > to maintain content that can't be built for current releases, so this
> > is problematic.
> >
>
> We could permit it, sure. Modularity has facilities for implementing
> it already, whereas non-modular would need to grow new rules for the
> Message Tagging Service or do a lot of manual tagging.
>
> > Also, if we did this, we should drop the DistTag, because it confuses
> > people otherwise.
> >
>
> Could you explain that?

People regularly look at NEVRAs to identify whether there are
"broken"/"old" packages to clean up, and when you see .fc29 installed
on an otherwise .fc31 system, it looks like something has gone wrong.
If packages are intended to be built once and pushed to multiple
releases, it shouldn't have the DistTag suffix.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Miro Hrončok]

On 17. 10. 19 14:08, Stephen Gallagher wrote:

On Thu, Oct 17, 2019 at 7:53 AM Miro Hrončok  wrote:

That was a representative example. I came up with it at 11pm after a
long day. Don't read too much into the specifics. The point was that
builds may require newer or older software than we have available in
the non-modular buildroot, but don't require them at runtime.


Yes, they do. There are many ways to solve the problem, from compat packages
trough various tag-magic to buildroot overrides in custom side tags. We have had
this problem for years and we were able to deal with it quite well.



Please let's not confuse "we have figured out how to work around it by
making our packagers do a lot of tricky hacks" with "able to deal with
it quite well". This is a major issue for non-expert packagers. I
don't have statistics on how many people give up on a package because
of this issue, but I know *I* have done so (and I'd consider myself
more experienced than most people at packaging).

One of the (often un- or misinformed) major arguments people keep
using against Modularity is "it makes packaging harder!". This is one
place where it makes things *much* easier on the packagers. It's a
clear reduction in complexity.


The real problem here is: It makes things easier for packagers who opt-in to 
modularity at a cost of making things harder for packagers who don't.


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Stephen Gallagher]

On Thu, Oct 17, 2019 at 7:53 AM Miro Hrončok  wrote:
> > That was a representative example. I came up with it at 11pm after a
> > long day. Don't read too much into the specifics. The point was that
> > builds may require newer or older software than we have available in
> > the non-modular buildroot, but don't require them at runtime.
>
> Yes, they do. There are many ways to solve the problem, from compat packages
> trough various tag-magic to buildroot overrides in custom side tags. We have 
> had
> this problem for years and we were able to deal with it quite well.
>

Please let's not confuse "we have figured out how to work around it by
making our packagers do a lot of tricky hacks" with "able to deal with
it quite well". This is a major issue for non-expert packagers. I
don't have statistics on how many people give up on a package because
of this issue, but I know *I* have done so (and I'd consider myself
more experienced than most people at packaging).

One of the (often un- or misinformed) major arguments people keep
using against Modularity is "it makes packaging harder!". This is one
place where it makes things *much* easier on the packagers. It's a
clear reduction in complexity.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Miro Hrončok]

On 17. 10. 19 13:41, Stephen Gallagher wrote:

On Thu, Oct 17, 2019 at 5:17 AM Miro Hrončok  wrote:


On 17. 10. 19 2:41, Stephen Gallagher wrote:

For example, we might have packages whose buildsystem
still relies on Python 2 (WAF?) but doesn't require it at runtime.


We do have them. How does that relate to modularity at all?



That was a representative example. I came up with it at 11pm after a
long day. Don't read too much into the specifics. The point was that
builds may require newer or older software than we have available in
the non-modular buildroot, but don't require them at runtime.


Yes, they do. There are many ways to solve the problem, from compat packages 
trough various tag-magic to buildroot overrides in custom side tags. We have had 
this problem for years and we were able to deal with it quite well.



Similarly, the example of "build on Rawhide, run anywhere" was
backwards. I should have said "build on oldest supported Fedora, carry
through".


As others have pointed out, this is technically possible without modules and we 
don't do it because we are (arguably rightfully) obsessed with rebuilds.


There is no technical problem to tag a .fc29 build into f29, f30, f31 and f32 
(except that currently, bodhi will not let you do that, so admin permissions are 
probably necessary; however this is already possible for anybody for branched 
(prior to bodhi enablement) + rawhide).


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Stephen Gallagher]

On Thu, Oct 17, 2019 at 7:48 AM Neal Gompa  wrote:
>
> On Thu, Oct 17, 2019 at 7:42 AM Stephen Gallagher  wrote:
> >
> > Similarly, the example of "build on Rawhide, run anywhere" was
> > backwards. I should have said "build on oldest supported Fedora, carry
> > through".
>
> Modules currently fail at this because they have a platform
> dependency.

I meant to address that in an earlier email when I saw you mention
that. It was wrong then, too.

Most modules *specify* a platform dependency, but they *do not have to*.

Take a look at the maven module, for example:
https://src.fedoraproject.org/modules/maven/blob/3.5/f/maven.yaml

You will notice that it specifies F29 as a build requirement but
specifies '[ ]' for the platform: this means "Build it on F29, run it
anywhere".

> And we could *easily* permit it for non-modular packages
> if we actually wanted it. I think we've generally said we don't want
> to maintain content that can't be built for current releases, so this
> is problematic.
>

We could permit it, sure. Modularity has facilities for implementing
it already, whereas non-modular would need to grow new rules for the
Message Tagging Service or do a lot of manual tagging.

> Also, if we did this, we should drop the DistTag, because it confuses
> people otherwise.
>

Could you explain that?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Neal Gompa]

On Thu, Oct 17, 2019 at 7:42 AM Stephen Gallagher  wrote:
>
> Similarly, the example of "build on Rawhide, run anywhere" was
> backwards. I should have said "build on oldest supported Fedora, carry
> through".

Modules currently fail at this because they have a platform
dependency. And we could *easily* permit it for non-modular packages
if we actually wanted it. I think we've generally said we don't want
to maintain content that can't be built for current releases, so this
is problematic.

Also, if we did this, we should drop the DistTag, because it confuses
people otherwise.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Stephen Gallagher]

On Thu, Oct 17, 2019 at 5:27 AM Jakub Cajka  wrote:

> For Go this is oversimplification and common misconception(go built binaries 
> are not uncommonly dynamically linked in the "C/ELF" sense(glibc,...) and 
> statically linked in Go sense). It might work for some selected(maybe even 
> most) Go based packages, but not always(new glibc,...) and not universally.

See my other reply. These were meant to be stand-in examples. Don't
read too much into the specifics; I wrote them when I was tired.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Stephen Gallagher]

On Thu, Oct 17, 2019 at 5:17 AM Miro Hrončok  wrote:
>
> On 17. 10. 19 2:41, Stephen Gallagher wrote:
> > For example, we might have packages whose buildsystem
> > still relies on Python 2 (WAF?) but doesn't require it at runtime.
>
> We do have them. How does that relate to modularity at all?
>

That was a representative example. I came up with it at 11pm after a
long day. Don't read too much into the specifics. The point was that
builds may require newer or older software than we have available in
the non-modular buildroot, but don't require them at runtime.

Similarly, the example of "build on Rawhide, run anywhere" was
backwards. I should have said "build on oldest supported Fedora, carry
through".
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Jakub Cajka]

- Original Message -
> From: "Stephen Gallagher" 
> To: "Development discussions related to Fedora" 
> 
> Sent: Thursday, October 17, 2019 2:41:28 AM
> Subject: Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular 
> Buildroot
> 
> > > (And how would restricting default streams to only be able to depend on
> > > default streams change things?)
> >
> > It would solve the version conflicts issue, so it makes a lot of sense, but
> > at that point, why not require the default versions to just be non-modular
> > instead? The main argument for using default streams was that they can
> > depend on non-default streams of other modules. So if we disallow this
> > (which I think makes sense), we may as well disallow default streams
> > entirely and simplify things for everyone. (We would just need a short-term
> > workaround for the default streams that exist now. The problem would be
> > gone
> > in the long run.)
> >
> 
> There's still a case that you haven't considered, which is things that
> work at runtime as a default but cannot *build* against the default
> set of packages. For example, we might have packages whose buildsystem
> still relies on Python 2 (WAF?) but doesn't require it at runtime.
> There might be packages that haven't yet migrated to a new,
> backwards-incompatible change to Docbook for generating documentation.
> Or packages that only build properly with a newer version of golang
> than shipped in that release, or any of a thousand other examples that
> are easily solved by build-time-only content and dependencies in
> module streams.

In current context(1.X) package that doesn't build with new version of Go has 
usually serious issue(s) on the side of the package/upstream. Fortunately this 
is really uncommon in my experience(even for packages/upstreams that fear/think 
that they will break with newer version of Go).

> 
> Also there are yet other packages (such as in the Go and Rust
> ecosystems) that could simply be built once on Rawhide with the latest
> compiler and shipped to each of the other releases without needing a
> rebuild because they are statically linked. Modules allow this, basic
> RPMs not so much.

For Go this is oversimplification and common misconception(go built binaries 
are not uncommonly dynamically linked in the "C/ELF" sense(glibc,...) and 
statically linked in Go sense). It might work for some selected(maybe even 
most) Go based packages, but not always(new glibc,...) and not universally. 

JC

> 
> So even if we eliminate the version conflicts issue by restricting
> what comprises a default stream, there are other benefits to module
> default streams.
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Miro Hrončok]

On 17. 10. 19 2:41, Stephen Gallagher wrote:

For example, we might have packages whose buildsystem
still relies on Python 2 (WAF?) but doesn't require it at runtime.


We do have them. How does that relate to modularity at all?

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Miro Hrončok]

On 16. 10. 19 19:28, Stephen Gallagher wrote:

1) This will be solved by the new Koji/MBS feature that we've
codenamed "Ursa Prime" (as a replacement for the original "Ursa Major"
tool that was built for RHEL 8). Unlike its predecessor, it requires
no additional daemon service running to handle things. We will create
a buildroot compose for each release that is created from the set of
packages available from all of the default streams and both Koji (in
infra) and mock (on packager systems) will be able to consume this.
Koji will behave the same as mock, where it will rely on libdnf's
default module stream handling to populate the buildroot, so we won't
have to worry about disparities between the local and infrastructure
packager experiences.


Thank You! Please describe this in the change proposal.

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Kevin Kofler]

Stephen Gallagher wrote:
> Not necessarily. It may be that we have to content ourselves with some
> software always requiring a module enablement to use it. For example,
> I maintain a module for Review Board, a Django-based code review tool.
> For complicated reasons, it cannot run against Django newer than 1.6
> (the Review Board upstream maintains the 1.6 stream of Django for
> security patches). If we decide that deps of default streams are bound
> to the same rules as default streams (or, alternately, Matthew's
> proposal of requiring only depending on default streams), then the
> necessary outcome is that Review Board can only be available via
> non-default stream (since it depends on django:1.6).

The user-friendly approach for that is to use a parallel-installable 
compatibility package (with a suffixed Name, such as django1.6) instead of a 
module.

> There's still a case that you haven't considered, which is things that
> work at runtime as a default but cannot *build* against the default
> set of packages. For example, we might have packages whose buildsystem
> still relies on Python 2 (WAF?) but doesn't require it at runtime.

There too, the correct approach is to provide (continue providing) a python2 
compatibility package. (Even the python27 one the Python SIG has introduced 
in Rawhide might be enough, actually.) There is no technical reason for it 
being in a module.

> There might be packages that haven't yet migrated to a new,
> backwards-incompatible change to Docbook for generating documentation.
> Or packages that only build properly with a newer version of golang
> than shipped in that release, or any of a thousand other examples that
> are easily solved by build-time-only content and dependencies in
> module streams.

The preferred solution there is really to fix the package to build with the 
latest version, which is the way we have always handled such FTBFS issues. 
But in the worst case, we can just ship the last successful build until the 
FTBFS is fixed. I think we are giving way too much importance to FTBFS bugs.

> Also there are yet other packages (such as in the Go and Rust
> ecosystems) that could simply be built once on Rawhide with the latest
> compiler and shipped to each of the other releases without needing a
> rebuild because they are statically linked. Modules allow this, basic
> RPMs not so much.

As Neal Gompa hinted at, that is not a good idea because the build will be 
linked with the Rawhide glibc and, as a result, typically not run on the 
older releases.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Neal Gompa]

On Wed, Oct 16, 2019 at 8:42 PM Stephen Gallagher  wrote:
>
> On Wed, Oct 16, 2019 at 8:12 PM Kevin Kofler  wrote:
> >
> > Stephen Gallagher wrote:
> > > 3) We need to get the policy I described above written onto -stone
> > > tablets- the Packaging Guidelines and then we need to go and make any
> > > stream that isn't compliant with it a non-default stream.
> >
> > But then we need a policy that requires a default version (non-modular or at
> > least a default stream) to be available. Otherwise we end up with packages
> > that are not installable out of the box because they have no default version
> > at all.
> >
>
> Not necessarily. It may be that we have to content ourselves with some
> software always requiring a module enablement to use it. For example,
> I maintain a module for Review Board, a Django-based code review tool.
> For complicated reasons, it cannot run against Django newer than 1.6
> (the Review Board upstream maintains the 1.6 stream of Django for
> security patches). If we decide that deps of default streams are bound
> to the same rules as default streams (or, alternately, Matthew's
> proposal of requiring only depending on default streams), then the
> necessary outcome is that Review Board can only be available via
> non-default stream (since it depends on django:1.6).
>
> That said, I'm sure that we can come up with ways to make that more
> discoverable, but today we have limitations with how DNF is able to do
> `search` or `repoquery`... it can only manage this with default or
> enabled streams and it doesn't "see" the non-default streams. If we go
> this path, I'd want to hear some input (and get some help!) on
> improving the discovery experience.
>

A big chunk of the problem in fixing this will go away once this is
merged: https://github.com/rpm-software-management/libdnf/pull/814

> > Matthew Miller wrote:
> > > How would this act in the case where a default stream depends on a
> > > non-default stream?
> >
> > From a policy standpoint, that non-default stream then ought to be bound by
> > the same rules as default streams. But allowing a default stream to depend
> > on a non-default stream paves the way for version conflicts to happen, so I
> > am not convinced that it is a good idea to begin with.
> >
>
> With the notable caveat of the example above, I'm willing to accept
> this restriction in Fedora. We just need to acknowledge that it *will*
> mean that some software we provide is less discoverable. If that's
> deemed to be a worthwhile tradeoff, I have no problems with
> implementing it.
>

We need a way for module dependencies to not care of
default/non-default and handle this gracefully, even in transitions.

> > > (And how would restricting default streams to only be able to depend on
> > > default streams change things?)
> >
> > It would solve the version conflicts issue, so it makes a lot of sense, but
> > at that point, why not require the default versions to just be non-modular
> > instead? The main argument for using default streams was that they can
> > depend on non-default streams of other modules. So if we disallow this
> > (which I think makes sense), we may as well disallow default streams
> > entirely and simplify things for everyone. (We would just need a short-term
> > workaround for the default streams that exist now. The problem would be gone
> > in the long run.)
> >
>
> There's still a case that you haven't considered, which is things that
> work at runtime as a default but cannot *build* against the default
> set of packages. For example, we might have packages whose buildsystem
> still relies on Python 2 (WAF?) but doesn't require it at runtime.
> There might be packages that haven't yet migrated to a new,
> backwards-incompatible change to Docbook for generating documentation.
> Or packages that only build properly with a newer version of golang
> than shipped in that release, or any of a thousand other examples that
> are easily solved by build-time-only content and dependencies in
> module streams.
>
> Also there are yet other packages (such as in the Go and Rust
> ecosystems) that could simply be built once on Rawhide with the latest
> compiler and shipped to each of the other releases without needing a
> rebuild because they are statically linked. Modules allow this, basic
> RPMs not so much.
>

This is actually not true. Every language stack we have does some
degree of dynamic linking, even if it's only to glibc.

The *only* reason we don't allow non-modular RPMs to be built once in
rawhide and shipped to all releases without a rebuild is purely
policy. We can mechanically do that since Koji lets you multi-tag RPMs
into multiple collections.

> So even if we eliminate the version conflicts issue by restricting
> what comprises a default stream, there are other benefits to module
> default streams.

Ehh...



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- 

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Stephen Gallagher]

On Wed, Oct 16, 2019 at 8:12 PM Kevin Kofler  wrote:
>
> Stephen Gallagher wrote:
> > 3) We need to get the policy I described above written onto -stone
> > tablets- the Packaging Guidelines and then we need to go and make any
> > stream that isn't compliant with it a non-default stream.
>
> But then we need a policy that requires a default version (non-modular or at
> least a default stream) to be available. Otherwise we end up with packages
> that are not installable out of the box because they have no default version
> at all.
>

Not necessarily. It may be that we have to content ourselves with some
software always requiring a module enablement to use it. For example,
I maintain a module for Review Board, a Django-based code review tool.
For complicated reasons, it cannot run against Django newer than 1.6
(the Review Board upstream maintains the 1.6 stream of Django for
security patches). If we decide that deps of default streams are bound
to the same rules as default streams (or, alternately, Matthew's
proposal of requiring only depending on default streams), then the
necessary outcome is that Review Board can only be available via
non-default stream (since it depends on django:1.6).

That said, I'm sure that we can come up with ways to make that more
discoverable, but today we have limitations with how DNF is able to do
`search` or `repoquery`... it can only manage this with default or
enabled streams and it doesn't "see" the non-default streams. If we go
this path, I'd want to hear some input (and get some help!) on
improving the discovery experience.

> Matthew Miller wrote:
> > How would this act in the case where a default stream depends on a
> > non-default stream?
>
> From a policy standpoint, that non-default stream then ought to be bound by
> the same rules as default streams. But allowing a default stream to depend
> on a non-default stream paves the way for version conflicts to happen, so I
> am not convinced that it is a good idea to begin with.
>

With the notable caveat of the example above, I'm willing to accept
this restriction in Fedora. We just need to acknowledge that it *will*
mean that some software we provide is less discoverable. If that's
deemed to be a worthwhile tradeoff, I have no problems with
implementing it.

> > (And how would restricting default streams to only be able to depend on
> > default streams change things?)
>
> It would solve the version conflicts issue, so it makes a lot of sense, but
> at that point, why not require the default versions to just be non-modular
> instead? The main argument for using default streams was that they can
> depend on non-default streams of other modules. So if we disallow this
> (which I think makes sense), we may as well disallow default streams
> entirely and simplify things for everyone. (We would just need a short-term
> workaround for the default streams that exist now. The problem would be gone
> in the long run.)
>

There's still a case that you haven't considered, which is things that
work at runtime as a default but cannot *build* against the default
set of packages. For example, we might have packages whose buildsystem
still relies on Python 2 (WAF?) but doesn't require it at runtime.
There might be packages that haven't yet migrated to a new,
backwards-incompatible change to Docbook for generating documentation.
Or packages that only build properly with a newer version of golang
than shipped in that release, or any of a thousand other examples that
are easily solved by build-time-only content and dependencies in
module streams.

Also there are yet other packages (such as in the Go and Rust
ecosystems) that could simply be built once on Rawhide with the latest
compiler and shipped to each of the other releases without needing a
rebuild because they are statically linked. Modules allow this, basic
RPMs not so much.

So even if we eliminate the version conflicts issue by restricting
what comprises a default stream, there are other benefits to module
default streams.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Kevin Kofler]

Stephen Gallagher wrote:
> 3) We need to get the policy I described above written onto -stone
> tablets- the Packaging Guidelines and then we need to go and make any
> stream that isn't compliant with it a non-default stream.

But then we need a policy that requires a default version (non-modular or at 
least a default stream) to be available. Otherwise we end up with packages 
that are not installable out of the box because they have no default version 
at all.

Matthew Miller wrote:
> How would this act in the case where a default stream depends on a
> non-default stream?

From a policy standpoint, that non-default stream then ought to be bound by 
the same rules as default streams. But allowing a default stream to depend 
on a non-default stream paves the way for version conflicts to happen, so I 
am not convinced that it is a good idea to begin with.

> (And how would restricting default streams to only be able to depend on
> default streams change things?)

It would solve the version conflicts issue, so it makes a lot of sense, but 
at that point, why not require the default versions to just be non-modular 
instead? The main argument for using default streams was that they can 
depend on non-default streams of other modules. So if we disallow this 
(which I think makes sense), we may as well disallow default streams 
entirely and simplify things for everyone. (We would just need a short-term 
workaround for the default streams that exist now. The problem would be gone 
in the long run.)

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[John M. Harris Jr]

On Wednesday, October 16, 2019 2:30:21 PM MST Matthew Miller wrote:
> On Wed, Oct 16, 2019 at 04:30:32PM -0400, Stephen Gallagher wrote:
> 
> > The idea is that it would act exactly the same way that dnf on the
> > local system would act: if you builddep software from a stream that
> > requires a non-default stream, it would enable that non-default
> > stream.
> 
> 
> Ah, I see. Thanks!
> 
> 
> > > (And how would restricting default streams to only be able to depend on
> > > default streams change things?)
> > 
> > There wouldn't need to be a behavior change; this is all done at the
> > libdnf layer; it just means that there is less available software in
> > the buildroot (since we'd necessarily have to exclude anything that
> > would conflict).
> 
> 
> I still see "non default stream enabled by default stream" to be surprising
> behavior we should avoid. But perhaps I am making too big a hammer of it.
> :) 
> -- 
> Matthew Miller
> 
> Fedora Project Leader
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List
> Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List
> Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Realistically, I believe that default streams themselves are something we 
should avoid, if the cost is low, and it is. There are many users, probably 
the vast majority of users, that don't use Modularity. It's great to have the 
option available, but to force it upon them is really unfortunate.

Additionally, wouldn't a "default module" pulling in a "non-default module" 
cause mass breakage? I can only see that causing ABI breakage..

-- 
John M. Harris, Jr.
Splentity

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Matthew Miller]

On Wed, Oct 16, 2019 at 04:30:32PM -0400, Stephen Gallagher wrote:
> The idea is that it would act exactly the same way that dnf on the
> local system would act: if you builddep software from a stream that
> requires a non-default stream, it would enable that non-default
> stream.

Ah, I see. Thanks!

> > (And how would restricting default streams to only be able to depend on
> > default streams change things?)
> There wouldn't need to be a behavior change; this is all done at the
> libdnf layer; it just means that there is less available software in
> the buildroot (since we'd necessarily have to exclude anything that
> would conflict).

I still see "non default stream enabled by default stream" to be surprising
behavior we should avoid. But perhaps I am making too big a hammer of it. :)

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Stephen Gallagher]

On Wed, Oct 16, 2019 at 2:41 PM Matthew Miller  wrote:
>
> On Wed, Oct 16, 2019 at 01:28:37PM -0400, Stephen Gallagher wrote:
> > 1) This will be solved by the new Koji/MBS feature that we've
> > codenamed "Ursa Prime" (as a replacement for the original "Ursa Major"
> > tool that was built for RHEL 8). Unlike its predecessor, it requires
> > no additional daemon service running to handle things. We will create
> > a buildroot compose for each release that is created from the set of
> > packages available from all of the default streams and both Koji (in
> > infra) and mock (on packager systems) will be able to consume this.
> > Koji will behave the same as mock, where it will rely on libdnf's
> > default module stream handling to populate the buildroot, so we won't
> > have to worry about disparities between the local and infrastructure
> > packager experiences.
>
> How would this act in the case where a default stream depends on a
> non-default stream?
>

The idea is that it would act exactly the same way that dnf on the
local system would act: if you builddep software from a stream that
requires a non-default stream, it would enable that non-default
stream.

> (And how would restricting default streams to only be able to depend on
> default streams change things?)

There wouldn't need to be a behavior change; this is all done at the
libdnf layer; it just means that there is less available software in
the buildroot (since we'd necessarily have to exclude anything that
would conflict).
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Matthew Miller]

On Wed, Oct 16, 2019 at 01:28:37PM -0400, Stephen Gallagher wrote:
> 1) This will be solved by the new Koji/MBS feature that we've
> codenamed "Ursa Prime" (as a replacement for the original "Ursa Major"
> tool that was built for RHEL 8). Unlike its predecessor, it requires
> no additional daemon service running to handle things. We will create
> a buildroot compose for each release that is created from the set of
> packages available from all of the default streams and both Koji (in
> infra) and mock (on packager systems) will be able to consume this.
> Koji will behave the same as mock, where it will rely on libdnf's
> default module stream handling to populate the buildroot, so we won't
> have to worry about disparities between the local and infrastructure
> packager experiences.

How would this act in the case where a default stream depends on a
non-default stream?

(And how would restricting default streams to only be able to depend on
default streams change things?)


-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Stephen Gallagher]

On Wed, Oct 16, 2019 at 12:11 PM Adam Williamson
 wrote:
>
> On Tue, 2019-10-15 at 12:25 -0400, Neal Gompa wrote:
> >
> > > So: I'm on board with most of what you say here, but there's no need to
> > > say it means Modularity is "a failure". It means right now it's not
> > > entirely baked and we're realizing the concept needs extending and
> > > perhaps reworking a bit, just like we realized with the *first* cut at
> > > Modularity which we abandoned between a Beta release and a Final
> > > release. This is causing us to have to deal with some icky problems,
> > > but again, that's not *new*. We had to deal with some fairly icky
> > > problems when systemd showed up. We had to deal with some fairly icky
> > > problems when grub2 showed up. It's a process we've dealt with before.
> > > We know how to do it. We just need to hold our noses and fix the icky
> > > problems, and then sit down and think about the design issues that have
> > > become apparent in Modularity v2 through our actually implementing it
> > > and using it (which is what Fedora is for, remember!) and figure out
> > > how to address them in Modularity v3.
> >
> > Are we going to do a Modularity v3? Because people seemed to be
> > *really* reluctant to go down that path because it would break
> > compatibility with RHEL.
>
> Well, those are just the terms I use to think about it, they have no
> official validity. So in a sense it's a pointless semantic question.
> Let's put it this way: I'd be surprised if we don't wind up having to
> make significant changes/enhancements to the modularity rules and/or
> implementation to be able to resolve the issues we're dealing with
> right now (Stephen, in some of his replies, seems to agree) and to me
> it'd be valid to call that 'modularity v3'. Whether anyone else would
> call it that or not doesn't seem too important :)


I agree with everything Adam just said. I'd avoid the term "Modularity
v3" in general, because it sounds like a total redesign (which is what
the v1->v2 switch ended up being). So far, all of the things I've been
proposing have been done with an aim of not causing any additional
breakage and to ensure that an upgrade path exists.

That said, as we've rolled this out, new information has come up that
is requiring us to re-evaluate some of the original design decisions
(the most obvious being the "changing default streams" case). The two
different proposals for how to handle that have been offered up
because they provide an upgrade path to get out of the current
situation without significant manual intervention.

Also, I want to reiterate this point, because it is central: There are
three core issues from which most of the complaints in this thread
ultimately stem:

1) Modular default streams are not available to the non-modular
buildroot, so packages that want to build against them are forced to
either become modules or vanish from Fedora (or use complex build-time
workarounds).

2) Once a stream has been enabled on the system, users are bound to it
- even across upgrades between releases. Since the purpose of default
streams is to eliminate the need for users to understand module
interactions if they want to continue operating the way they did
pre-Modularity, they need to be able to follow stream changes on
upgrades to match the intended defaults for the new system.

3) The policy on default streams was not clearly defined and
communicated. The Modularity WG recently voted to affirm that all
artifacts installable from default streams of a module must obey all
of the rules of packages in non-modular Fedora (primarily Stable
Updates policy and the expectation that they are treated as supported
API for all purposes). This is the "Java/Maven Problem", where the
Java team created a maven module that included stripped-down
dependencies in the default stream, thus owning those package names
with unsupported content.


We have plans in place for how to deal with each of these critical
problems (without resorting to throwing it all away).

1) This will be solved by the new Koji/MBS feature that we've
codenamed "Ursa Prime" (as a replacement for the original "Ursa Major"
tool that was built for RHEL 8). Unlike its predecessor, it requires
no additional daemon service running to handle things. We will create
a buildroot compose for each release that is created from the set of
packages available from all of the default streams and both Koji (in
infra) and mock (on packager systems) will be able to consume this.
Koji will behave the same as mock, where it will rely on libdnf's
default module stream handling to populate the buildroot, so we won't
have to worry about disparities between the local and infrastructure
packager experiences.

2) There is an entire email thread [1] dedicated to how we are going
to solve this. I won't repeat it here.

3) We need to get the policy I described above written onto -stone
tablets- the Packaging Guidelines and then we need to go and make any
stream that 

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Adam Williamson]

On Tue, 2019-10-15 at 12:25 -0400, Neal Gompa wrote:
> 
> > So: I'm on board with most of what you say here, but there's no need to
> > say it means Modularity is "a failure". It means right now it's not
> > entirely baked and we're realizing the concept needs extending and
> > perhaps reworking a bit, just like we realized with the *first* cut at
> > Modularity which we abandoned between a Beta release and a Final
> > release. This is causing us to have to deal with some icky problems,
> > but again, that's not *new*. We had to deal with some fairly icky
> > problems when systemd showed up. We had to deal with some fairly icky
> > problems when grub2 showed up. It's a process we've dealt with before.
> > We know how to do it. We just need to hold our noses and fix the icky
> > problems, and then sit down and think about the design issues that have
> > become apparent in Modularity v2 through our actually implementing it
> > and using it (which is what Fedora is for, remember!) and figure out
> > how to address them in Modularity v3.
> 
> Are we going to do a Modularity v3? Because people seemed to be
> *really* reluctant to go down that path because it would break
> compatibility with RHEL.

Well, those are just the terms I use to think about it, they have no
official validity. So in a sense it's a pointless semantic question.
Let's put it this way: I'd be surprised if we don't wind up having to
make significant changes/enhancements to the modularity rules and/or
implementation to be able to resolve the issues we're dealing with
right now (Stephen, in some of his replies, seems to agree) and to me
it'd be valid to call that 'modularity v3'. Whether anyone else would
call it that or not doesn't seem too important :)
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Kevin Kofler]

Matthew Miller wrote:
> Upgrades need to work, though. And they need to work regardless of whether
> we ban default modules or not. So, given that, I'm not _really_ seeing big
> differences in practice for the user beteen these two proposals, and the
> one (no default streams) negates one of the whole intended advantages of
> the entire thing.
> 
> What am I missing?

I don't see how requiring a non-modular default version negates a central 
advantage of Modularity. You can still offer all the versions of your module 
and get all the benefits of Modularity. You just cannot force the module 
onto all users, because that turned out to be causing many more problems 
than it solves.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Kevin Kofler]

Joe Orton wrote:
> If you don't want to ship any of your packages as modules I think that's
> great and you should continue doing that.  On the other hand, I want to
> move a bunch of my packages to module-only because I think I can do a
> better job serving Fedora users that way.

How so? That is the big question that you are leaving unanswered: In what 
way do you think you can do a better job serving Fedora users with module-
only packages than by also delivering a non-modular version (along with the 
modular version for those who actually want it)?

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Ken Dreyer]

On Tue, Oct 15, 2019 at 10:13 AM Adam Williamson
 wrote:
> What's happening right now is the process of us trying something out
> and finding out where the problems are. That's what happens when you
> invent new stuff, it's harder than just carrying on doing the old
> stuff.

I agree Adam.

I think Neal's summary helps and I appreciate it. I agree it's
negative, but I think there is one additional aspect from my point of
view: There are some proponents of Modularity that keep pitching the
idea to me that everything I do needs to go into a module, like,
yesterday. Maybe we can tone that messaging down a bit around Fedora
for a while.

- Ken


- Ken
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Robbie Harwood]

Tomasz Torcz  writes:

> On Tue, Oct 15, 2019 at 01:56:11PM -0400, Robbie Harwood wrote:
>> Matthew Miller  writes:
>> 
>> > On Tue, Oct 15, 2019 at 12:36:15PM +0200, Miro Hrončok wrote:
>> >>> As package maintainers we all make technical decisions which have
>> >>> significant impact on our users every day - whether that's in the
>> >>> choice of defaults, choice of build flags, or whatever.  Honestly
>> >>> delivering as modules-vs-non-modules is a completely trivial issue
>> >>> compared to most of the stuff I spend time on.  If "yum install X"
>> >>> still works most people just don't care about the RPM/dnf/repo
>> >>> mechanics behind that.
>> >> 
>> >> Except it works only half way. The installation works. Later,
>> >> dependencies are broken. Upgrades are broken. "yum remove X" does not
>> >> undo the action completely.
>> >> 
>> >> The main issue is: user just enabled a module without doing it
>> >> explicitly. The user needs to know how to handle modules in order to
>> >> recover.
>> >
>> > I never expect "yum remove X" to be the inverse of "yum install
>> > X". DNF's magical leaf tracking makes it a bit more so, but not
>> > exactly. So, I don't think we should make that a very high priority
>> > concern (although if we can improve it, so much the better).
>> 
>> I don't think it's an unreasonable expectation, especially for those
>> coming from APT land (Debian, Ubuntu) where `apt install foo` *is* the
>> inverse operation of `apt remove foo`.
>
>   It isn't, you need to supplement it with `apt autoremove` to get rid
> of auto-installed dependencies.
>   We have `dnf history undo X` to invert installation command.

Ah, you're right.  It is the case with aptitude, but not with apt or
apt-get.  Sorry for the noise.

Thanks,
--Robbie


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Tomasz Torcz]

On Tue, Oct 15, 2019 at 01:56:11PM -0400, Robbie Harwood wrote:
> Matthew Miller  writes:
> 
> > On Tue, Oct 15, 2019 at 12:36:15PM +0200, Miro Hrončok wrote:
> >>> As package maintainers we all make technical decisions which have
> >>> significant impact on our users every day - whether that's in the
> >>> choice of defaults, choice of build flags, or whatever.  Honestly
> >>> delivering as modules-vs-non-modules is a completely trivial issue
> >>> compared to most of the stuff I spend time on.  If "yum install X"
> >>> still works most people just don't care about the RPM/dnf/repo
> >>> mechanics behind that.
> >> 
> >> Except it works only half way. The installation works. Later,
> >> dependencies are broken. Upgrades are broken. "yum remove X" does not
> >> undo the action completely.
> >> 
> >> The main issue is: user just enabled a module without doing it
> >> explicitly. The user needs to know how to handle modules in order to
> >> recover.
> >
> > I never expect "yum remove X" to be the inverse of "yum install
> > X". DNF's magical leaf tracking makes it a bit more so, but not
> > exactly. So, I don't think we should make that a very high priority
> > concern (although if we can improve it, so much the better).
> 
> I don't think it's an unreasonable expectation, especially for those
> coming from APT land (Debian, Ubuntu) where `apt install foo` *is* the
> inverse operation of `apt remove foo`.

  It isn't, you need to supplement it with `apt autoremove` to get rid
of auto-installed dependencies.
  We have `dnf history undo X` to invert installation command.


-- 
Tomasz   .. oo o.   oo o. .o   .o o. o. oo o.   ..
Torcz.. .o .o   .o .o oo   oo .o .. .. oo   oo
o.o.o.   .o .. o.   o. o. o.   o. o. oo .. ..   o.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Robbie Harwood]

Matthew Miller  writes:

> On Tue, Oct 15, 2019 at 12:36:15PM +0200, Miro Hrončok wrote:
>>> As package maintainers we all make technical decisions which have
>>> significant impact on our users every day - whether that's in the
>>> choice of defaults, choice of build flags, or whatever.  Honestly
>>> delivering as modules-vs-non-modules is a completely trivial issue
>>> compared to most of the stuff I spend time on.  If "yum install X"
>>> still works most people just don't care about the RPM/dnf/repo
>>> mechanics behind that.
>> 
>> Except it works only half way. The installation works. Later,
>> dependencies are broken. Upgrades are broken. "yum remove X" does not
>> undo the action completely.
>> 
>> The main issue is: user just enabled a module without doing it
>> explicitly. The user needs to know how to handle modules in order to
>> recover.
>
> I never expect "yum remove X" to be the inverse of "yum install
> X". DNF's magical leaf tracking makes it a bit more so, but not
> exactly. So, I don't think we should make that a very high priority
> concern (although if we can improve it, so much the better).

I don't think it's an unreasonable expectation, especially for those
coming from APT land (Debian, Ubuntu) where `apt install foo` *is* the
inverse operation of `apt remove foo`.

Thanks,
--Robbie


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Neal Gompa]

On Tue, Oct 15, 2019 at 12:52 PM Josh Boyer  wrote:
>
> On Tue, Oct 15, 2019 at 12:27 PM Neal Gompa  wrote:
> >
> > On Tue, Oct 15, 2019 at 12:13 PM Adam Williamson
> >  wrote:
> > >
> > > On Tue, 2019-10-15 at 11:35 -0400, Neal Gompa wrote:
> > > > On Tue, Oct 15, 2019 at 11:22 AM Matthew Miller
> > > >  wrote:
> > > > > On Tue, Oct 15, 2019 at 12:36:15PM +0200, Miro Hrončok wrote:
> > > > > > > As package maintainers we all make technical decisions which have
> > > > > > > significant impact on our users every day - whether that's in the 
> > > > > > > choice
> > > > > > > of defaults, choice of build flags, or whatever.  Honestly 
> > > > > > > delivering as
> > > > > > > modules-vs-non-modules is a completely trivial issue compared to 
> > > > > > > most of
> > > > > > > the stuff I spend time on.  If "yum install X" still works most 
> > > > > > > people
> > > > > > > just don't care about the RPM/dnf/repo mechanics behind that.
> > > > > >
> > > > > > Except it works only half way. The installation works. Later,
> > > > > > dependencies are broken. Upgrades are broken. "yum remove X" does
> > > > > > not undo the action completely.
> > > > > >
> > > > > > The main issue is: user just enabled a module without doing it
> > > > > > explicitly. The user needs to know how to handle modules in order to
> > > > > > recover.
> > > > >
> > > > > I never expect "yum remove X" to be the inverse of "yum install X". 
> > > > > DNF's
> > > > > magical leaf tracking makes it a bit more so, but not exactly. So, I 
> > > > > don't
> > > > > think we should make that a very high priority concern (although if 
> > > > > we can
> > > > > improve it, so much the better).
> > > > >
> > > > > Upgrades need to work, though. And they need to work regardless of 
> > > > > whether
> > > > > we ban default modules or not. So, given that, I'm not _really_ 
> > > > > seeing big
> > > > > differences in practice for the user beteen these two proposals, and 
> > > > > the one
> > > > > (no default streams) negates one of the whole intended advantages of 
> > > > > the
> > > > > entire thing.
> > > > >
> > > > > What am I missing?
> > > > >
> > > >
> > > > Upgrades can never work without changing fundamental properties of 
> > > > modules.
> > > >
> > > > There are two traits of modules that break everything:
> > > > * Module activation is introduces a hard lock of content source
> > > > * There is no concept of module transitions
> > > >
> > > > In the RHEL world, these two qualities are not great, but they are
> > > > serviceable (RHEL 9 is going to hurt!). In the Fedora world, they make
> > > > having default modules essentially untenable. It could even be argued
> > > > that non-default modules are not serviceable either.
> > > >
> > > > There is no way in modularity to say that a module is being replaced
> > > > with another. There's also no way to say that a module is going away
> > > > and transition accordingly. Moreover, since modules have a strong
> > > > dependency on an abstract property that is defined as a consequence of
> > > > the system content (the "platform" module), it is not possible to have
> > > > orphan modules that are still binary compatible as you upgrade.
> > > > Finally, modularization and foregoing non-modular versions of content
> > > > has an implicit commitment that you *never* demodularize because there
> > > > is no module transition capabilities in the modularity technology.
> > > >
> > > > These flaws boil down to Fedora Modularity being a failure because the
> > > > technology does not account for the needs of Fedora as a distribution,
> > > > as a project, or as a community.
> > >
> > > I think this is actually a very cogent summary of the major problems
> > > we're dealing with in modularity right now, but it's *also* needlessly
> > > negative.
> > >
> > > Inventing new stuff is hard. You rarely get it right the first time.
> > > Speaking personally here I think it was a big mistake to do this second
> > > version of Modularity as fast as we did and drop it into RHEL 8 before
> > > it had been in Fedora for two minutes, that was an unforced error; but
> > > even if we hadn't done that, I'm not surprised the second cut at
> > > Modularity isn't perfect. The second cut at RPM wasn't perfect either,
> > > that's why we're on 4.x not 2.x. :)
> > >
> > > What's happening right now is the process of us trying something out
> > > and finding out where the problems are. That's what happens when you
> > > invent new stuff, it's harder than just carrying on doing the old
> > > stuff.
> > >
> > > So: I'm on board with most of what you say here, but there's no need to
> > > say it means Modularity is "a failure". It means right now it's not
> > > entirely baked and we're realizing the concept needs extending and
> > > perhaps reworking a bit, just like we realized with the *first* cut at
> > > Modularity which we abandoned between a Beta release and a Final
> > > release. This is causing us to have to 

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Josh Boyer]

On Tue, Oct 15, 2019 at 12:27 PM Neal Gompa  wrote:
>
> On Tue, Oct 15, 2019 at 12:13 PM Adam Williamson
>  wrote:
> >
> > On Tue, 2019-10-15 at 11:35 -0400, Neal Gompa wrote:
> > > On Tue, Oct 15, 2019 at 11:22 AM Matthew Miller
> > >  wrote:
> > > > On Tue, Oct 15, 2019 at 12:36:15PM +0200, Miro Hrončok wrote:
> > > > > > As package maintainers we all make technical decisions which have
> > > > > > significant impact on our users every day - whether that's in the 
> > > > > > choice
> > > > > > of defaults, choice of build flags, or whatever.  Honestly 
> > > > > > delivering as
> > > > > > modules-vs-non-modules is a completely trivial issue compared to 
> > > > > > most of
> > > > > > the stuff I spend time on.  If "yum install X" still works most 
> > > > > > people
> > > > > > just don't care about the RPM/dnf/repo mechanics behind that.
> > > > >
> > > > > Except it works only half way. The installation works. Later,
> > > > > dependencies are broken. Upgrades are broken. "yum remove X" does
> > > > > not undo the action completely.
> > > > >
> > > > > The main issue is: user just enabled a module without doing it
> > > > > explicitly. The user needs to know how to handle modules in order to
> > > > > recover.
> > > >
> > > > I never expect "yum remove X" to be the inverse of "yum install X". 
> > > > DNF's
> > > > magical leaf tracking makes it a bit more so, but not exactly. So, I 
> > > > don't
> > > > think we should make that a very high priority concern (although if we 
> > > > can
> > > > improve it, so much the better).
> > > >
> > > > Upgrades need to work, though. And they need to work regardless of 
> > > > whether
> > > > we ban default modules or not. So, given that, I'm not _really_ seeing 
> > > > big
> > > > differences in practice for the user beteen these two proposals, and 
> > > > the one
> > > > (no default streams) negates one of the whole intended advantages of the
> > > > entire thing.
> > > >
> > > > What am I missing?
> > > >
> > >
> > > Upgrades can never work without changing fundamental properties of 
> > > modules.
> > >
> > > There are two traits of modules that break everything:
> > > * Module activation is introduces a hard lock of content source
> > > * There is no concept of module transitions
> > >
> > > In the RHEL world, these two qualities are not great, but they are
> > > serviceable (RHEL 9 is going to hurt!). In the Fedora world, they make
> > > having default modules essentially untenable. It could even be argued
> > > that non-default modules are not serviceable either.
> > >
> > > There is no way in modularity to say that a module is being replaced
> > > with another. There's also no way to say that a module is going away
> > > and transition accordingly. Moreover, since modules have a strong
> > > dependency on an abstract property that is defined as a consequence of
> > > the system content (the "platform" module), it is not possible to have
> > > orphan modules that are still binary compatible as you upgrade.
> > > Finally, modularization and foregoing non-modular versions of content
> > > has an implicit commitment that you *never* demodularize because there
> > > is no module transition capabilities in the modularity technology.
> > >
> > > These flaws boil down to Fedora Modularity being a failure because the
> > > technology does not account for the needs of Fedora as a distribution,
> > > as a project, or as a community.
> >
> > I think this is actually a very cogent summary of the major problems
> > we're dealing with in modularity right now, but it's *also* needlessly
> > negative.
> >
> > Inventing new stuff is hard. You rarely get it right the first time.
> > Speaking personally here I think it was a big mistake to do this second
> > version of Modularity as fast as we did and drop it into RHEL 8 before
> > it had been in Fedora for two minutes, that was an unforced error; but
> > even if we hadn't done that, I'm not surprised the second cut at
> > Modularity isn't perfect. The second cut at RPM wasn't perfect either,
> > that's why we're on 4.x not 2.x. :)
> >
> > What's happening right now is the process of us trying something out
> > and finding out where the problems are. That's what happens when you
> > invent new stuff, it's harder than just carrying on doing the old
> > stuff.
> >
> > So: I'm on board with most of what you say here, but there's no need to
> > say it means Modularity is "a failure". It means right now it's not
> > entirely baked and we're realizing the concept needs extending and
> > perhaps reworking a bit, just like we realized with the *first* cut at
> > Modularity which we abandoned between a Beta release and a Final
> > release. This is causing us to have to deal with some icky problems,
> > but again, that's not *new*. We had to deal with some fairly icky
> > problems when systemd showed up. We had to deal with some fairly icky
> > problems when grub2 showed up. It's a process we've dealt with before.
> > We know 

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Neal Gompa]

On Tue, Oct 15, 2019 at 12:13 PM Adam Williamson
 wrote:
>
> On Tue, 2019-10-15 at 11:35 -0400, Neal Gompa wrote:
> > On Tue, Oct 15, 2019 at 11:22 AM Matthew Miller
> >  wrote:
> > > On Tue, Oct 15, 2019 at 12:36:15PM +0200, Miro Hrončok wrote:
> > > > > As package maintainers we all make technical decisions which have
> > > > > significant impact on our users every day - whether that's in the 
> > > > > choice
> > > > > of defaults, choice of build flags, or whatever.  Honestly delivering 
> > > > > as
> > > > > modules-vs-non-modules is a completely trivial issue compared to most 
> > > > > of
> > > > > the stuff I spend time on.  If "yum install X" still works most people
> > > > > just don't care about the RPM/dnf/repo mechanics behind that.
> > > >
> > > > Except it works only half way. The installation works. Later,
> > > > dependencies are broken. Upgrades are broken. "yum remove X" does
> > > > not undo the action completely.
> > > >
> > > > The main issue is: user just enabled a module without doing it
> > > > explicitly. The user needs to know how to handle modules in order to
> > > > recover.
> > >
> > > I never expect "yum remove X" to be the inverse of "yum install X". DNF's
> > > magical leaf tracking makes it a bit more so, but not exactly. So, I don't
> > > think we should make that a very high priority concern (although if we can
> > > improve it, so much the better).
> > >
> > > Upgrades need to work, though. And they need to work regardless of whether
> > > we ban default modules or not. So, given that, I'm not _really_ seeing big
> > > differences in practice for the user beteen these two proposals, and the 
> > > one
> > > (no default streams) negates one of the whole intended advantages of the
> > > entire thing.
> > >
> > > What am I missing?
> > >
> >
> > Upgrades can never work without changing fundamental properties of modules.
> >
> > There are two traits of modules that break everything:
> > * Module activation is introduces a hard lock of content source
> > * There is no concept of module transitions
> >
> > In the RHEL world, these two qualities are not great, but they are
> > serviceable (RHEL 9 is going to hurt!). In the Fedora world, they make
> > having default modules essentially untenable. It could even be argued
> > that non-default modules are not serviceable either.
> >
> > There is no way in modularity to say that a module is being replaced
> > with another. There's also no way to say that a module is going away
> > and transition accordingly. Moreover, since modules have a strong
> > dependency on an abstract property that is defined as a consequence of
> > the system content (the "platform" module), it is not possible to have
> > orphan modules that are still binary compatible as you upgrade.
> > Finally, modularization and foregoing non-modular versions of content
> > has an implicit commitment that you *never* demodularize because there
> > is no module transition capabilities in the modularity technology.
> >
> > These flaws boil down to Fedora Modularity being a failure because the
> > technology does not account for the needs of Fedora as a distribution,
> > as a project, or as a community.
>
> I think this is actually a very cogent summary of the major problems
> we're dealing with in modularity right now, but it's *also* needlessly
> negative.
>
> Inventing new stuff is hard. You rarely get it right the first time.
> Speaking personally here I think it was a big mistake to do this second
> version of Modularity as fast as we did and drop it into RHEL 8 before
> it had been in Fedora for two minutes, that was an unforced error; but
> even if we hadn't done that, I'm not surprised the second cut at
> Modularity isn't perfect. The second cut at RPM wasn't perfect either,
> that's why we're on 4.x not 2.x. :)
>
> What's happening right now is the process of us trying something out
> and finding out where the problems are. That's what happens when you
> invent new stuff, it's harder than just carrying on doing the old
> stuff.
>
> So: I'm on board with most of what you say here, but there's no need to
> say it means Modularity is "a failure". It means right now it's not
> entirely baked and we're realizing the concept needs extending and
> perhaps reworking a bit, just like we realized with the *first* cut at
> Modularity which we abandoned between a Beta release and a Final
> release. This is causing us to have to deal with some icky problems,
> but again, that's not *new*. We had to deal with some fairly icky
> problems when systemd showed up. We had to deal with some fairly icky
> problems when grub2 showed up. It's a process we've dealt with before.
> We know how to do it. We just need to hold our noses and fix the icky
> problems, and then sit down and think about the design issues that have
> become apparent in Modularity v2 through our actually implementing it
> and using it (which is what Fedora is for, remember!) and figure out
> how to address them in 

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Adam Williamson]

On Tue, 2019-10-15 at 11:35 -0400, Neal Gompa wrote:
> On Tue, Oct 15, 2019 at 11:22 AM Matthew Miller
>  wrote:
> > On Tue, Oct 15, 2019 at 12:36:15PM +0200, Miro Hrončok wrote:
> > > > As package maintainers we all make technical decisions which have
> > > > significant impact on our users every day - whether that's in the choice
> > > > of defaults, choice of build flags, or whatever.  Honestly delivering as
> > > > modules-vs-non-modules is a completely trivial issue compared to most of
> > > > the stuff I spend time on.  If "yum install X" still works most people
> > > > just don't care about the RPM/dnf/repo mechanics behind that.
> > > 
> > > Except it works only half way. The installation works. Later,
> > > dependencies are broken. Upgrades are broken. "yum remove X" does
> > > not undo the action completely.
> > > 
> > > The main issue is: user just enabled a module without doing it
> > > explicitly. The user needs to know how to handle modules in order to
> > > recover.
> > 
> > I never expect "yum remove X" to be the inverse of "yum install X". DNF's
> > magical leaf tracking makes it a bit more so, but not exactly. So, I don't
> > think we should make that a very high priority concern (although if we can
> > improve it, so much the better).
> > 
> > Upgrades need to work, though. And they need to work regardless of whether
> > we ban default modules or not. So, given that, I'm not _really_ seeing big
> > differences in practice for the user beteen these two proposals, and the one
> > (no default streams) negates one of the whole intended advantages of the
> > entire thing.
> > 
> > What am I missing?
> > 
> 
> Upgrades can never work without changing fundamental properties of modules.
> 
> There are two traits of modules that break everything:
> * Module activation is introduces a hard lock of content source
> * There is no concept of module transitions
> 
> In the RHEL world, these two qualities are not great, but they are
> serviceable (RHEL 9 is going to hurt!). In the Fedora world, they make
> having default modules essentially untenable. It could even be argued
> that non-default modules are not serviceable either.
> 
> There is no way in modularity to say that a module is being replaced
> with another. There's also no way to say that a module is going away
> and transition accordingly. Moreover, since modules have a strong
> dependency on an abstract property that is defined as a consequence of
> the system content (the "platform" module), it is not possible to have
> orphan modules that are still binary compatible as you upgrade.
> Finally, modularization and foregoing non-modular versions of content
> has an implicit commitment that you *never* demodularize because there
> is no module transition capabilities in the modularity technology.
> 
> These flaws boil down to Fedora Modularity being a failure because the
> technology does not account for the needs of Fedora as a distribution,
> as a project, or as a community.

I think this is actually a very cogent summary of the major problems
we're dealing with in modularity right now, but it's *also* needlessly
negative.

Inventing new stuff is hard. You rarely get it right the first time.
Speaking personally here I think it was a big mistake to do this second
version of Modularity as fast as we did and drop it into RHEL 8 before
it had been in Fedora for two minutes, that was an unforced error; but
even if we hadn't done that, I'm not surprised the second cut at
Modularity isn't perfect. The second cut at RPM wasn't perfect either,
that's why we're on 4.x not 2.x. :)

What's happening right now is the process of us trying something out
and finding out where the problems are. That's what happens when you
invent new stuff, it's harder than just carrying on doing the old
stuff.

So: I'm on board with most of what you say here, but there's no need to
say it means Modularity is "a failure". It means right now it's not
entirely baked and we're realizing the concept needs extending and
perhaps reworking a bit, just like we realized with the *first* cut at
Modularity which we abandoned between a Beta release and a Final
release. This is causing us to have to deal with some icky problems,
but again, that's not *new*. We had to deal with some fairly icky
problems when systemd showed up. We had to deal with some fairly icky
problems when grub2 showed up. It's a process we've dealt with before.
We know how to do it. We just need to hold our noses and fix the icky
problems, and then sit down and think about the design issues that have
become apparent in Modularity v2 through our actually implementing it
and using it (which is what Fedora is for, remember!) and figure out
how to address them in Modularity v3.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- 

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Neal Gompa]

On Tue, Oct 15, 2019 at 11:22 AM Matthew Miller
 wrote:
>
> On Tue, Oct 15, 2019 at 12:36:15PM +0200, Miro Hrončok wrote:
> > >As package maintainers we all make technical decisions which have
> > >significant impact on our users every day - whether that's in the choice
> > >of defaults, choice of build flags, or whatever.  Honestly delivering as
> > >modules-vs-non-modules is a completely trivial issue compared to most of
> > >the stuff I spend time on.  If "yum install X" still works most people
> > >just don't care about the RPM/dnf/repo mechanics behind that.
> >
> > Except it works only half way. The installation works. Later,
> > dependencies are broken. Upgrades are broken. "yum remove X" does
> > not undo the action completely.
> >
> > The main issue is: user just enabled a module without doing it
> > explicitly. The user needs to know how to handle modules in order to
> > recover.
>
> I never expect "yum remove X" to be the inverse of "yum install X". DNF's
> magical leaf tracking makes it a bit more so, but not exactly. So, I don't
> think we should make that a very high priority concern (although if we can
> improve it, so much the better).
>
> Upgrades need to work, though. And they need to work regardless of whether
> we ban default modules or not. So, given that, I'm not _really_ seeing big
> differences in practice for the user beteen these two proposals, and the one
> (no default streams) negates one of the whole intended advantages of the
> entire thing.
>
> What am I missing?
>

Upgrades can never work without changing fundamental properties of modules.

There are two traits of modules that break everything:
* Module activation is introduces a hard lock of content source
* There is no concept of module transitions

In the RHEL world, these two qualities are not great, but they are
serviceable (RHEL 9 is going to hurt!). In the Fedora world, they make
having default modules essentially untenable. It could even be argued
that non-default modules are not serviceable either.

There is no way in modularity to say that a module is being replaced
with another. There's also no way to say that a module is going away
and transition accordingly. Moreover, since modules have a strong
dependency on an abstract property that is defined as a consequence of
the system content (the "platform" module), it is not possible to have
orphan modules that are still binary compatible as you upgrade.
Finally, modularization and foregoing non-modular versions of content
has an implicit commitment that you *never* demodularize because there
is no module transition capabilities in the modularity technology.

These flaws boil down to Fedora Modularity being a failure because the
technology does not account for the needs of Fedora as a distribution,
as a project, or as a community.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Matthew Miller]

On Tue, Oct 15, 2019 at 12:36:15PM +0200, Miro Hrončok wrote:
> >As package maintainers we all make technical decisions which have
> >significant impact on our users every day - whether that's in the choice
> >of defaults, choice of build flags, or whatever.  Honestly delivering as
> >modules-vs-non-modules is a completely trivial issue compared to most of
> >the stuff I spend time on.  If "yum install X" still works most people
> >just don't care about the RPM/dnf/repo mechanics behind that.
> 
> Except it works only half way. The installation works. Later,
> dependencies are broken. Upgrades are broken. "yum remove X" does
> not undo the action completely.
> 
> The main issue is: user just enabled a module without doing it
> explicitly. The user needs to know how to handle modules in order to
> recover.

I never expect "yum remove X" to be the inverse of "yum install X". DNF's
magical leaf tracking makes it a bit more so, but not exactly. So, I don't
think we should make that a very high priority concern (although if we can
improve it, so much the better).

Upgrades need to work, though. And they need to work regardless of whether
we ban default modules or not. So, given that, I'm not _really_ seeing big
differences in practice for the user beteen these two proposals, and the one
(no default streams) negates one of the whole intended advantages of the
entire thing.

What am I missing?

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Miro Hrončok]

On 15. 10. 19 12:10, Joe Orton wrote:

On Mon, Oct 14, 2019 at 11:44:46PM +0200, Kevin Kofler wrote:

The net result of this proposed Change for the end user is still the same as
the status quo: They have to use modules whether they want to or not, the
choice is taken away from them. And while the default stream approach tries
to hide Modularity from the users (and with this proposed Change, also from
the packagers of dependent packages), the abstraction is leaky, as
evidenced, e.g., by the libgit2 upgrade blocker.


Yup, it is certainly not perfect and we'll have to work through those
kinds of issues carefully as Stephen & others are doing.  I am very
confident in trusting Fedora developers to make sensible choices for our
users, though - including whether we should make packages module-only.

As package maintainers we all make technical decisions which have
significant impact on our users every day - whether that's in the choice
of defaults, choice of build flags, or whatever.  Honestly delivering as
modules-vs-non-modules is a completely trivial issue compared to most of
the stuff I spend time on.  If "yum install X" still works most people
just don't care about the RPM/dnf/repo mechanics behind that.


Except it works only half way. The installation works. Later, dependencies are 
broken. Upgrades are broken. "yum remove X" does not undo the action completely.


The main issue is: user just enabled a module without doing it explicitly. The 
user needs to know how to handle modules in order to recover.


We are also Fedora developers and we are trying to do the sensible choice here.

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Joe Orton]

On Mon, Oct 14, 2019 at 11:44:46PM +0200, Kevin Kofler wrote:
> The net result of this proposed Change for the end user is still the same as 
> the status quo: They have to use modules whether they want to or not, the 
> choice is taken away from them. And while the default stream approach tries 
> to hide Modularity from the users (and with this proposed Change, also from 
> the packagers of dependent packages), the abstraction is leaky, as 
> evidenced, e.g., by the libgit2 upgrade blocker.

Yup, it is certainly not perfect and we'll have to work through those 
kinds of issues carefully as Stephen & others are doing.  I am very 
confident in trusting Fedora developers to make sensible choices for our 
users, though - including whether we should make packages module-only.

As package maintainers we all make technical decisions which have 
significant impact on our users every day - whether that's in the choice 
of defaults, choice of build flags, or whatever.  Honestly delivering as 
modules-vs-non-modules is a completely trivial issue compared to most of 
the stuff I spend time on.  If "yum install X" still works most people 
just don't care about the RPM/dnf/repo mechanics behind that.

If you don't want to ship any of your packages as modules I think that's 
great and you should continue doing that.  On the other hand, I want to 
move a bunch of my packages to module-only because I think I can do a 
better job serving Fedora users that way.

Regards, Joe
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Kevin Kofler]

Joe Orton wrote:
> I find myself a bit reluctant to write this mail because the language
> others are using in this thread is fairly ugly for a technical
> discussion in an open source project - about "forcing" people to develop
> packages in a certain way, "teaching them a lesson" etc.  Please calm it
> down & have some respect for technical decisions of other developers.

I am sorry if my wording irritated you. However, I would like to point out 
that…

> For some people here it is clear they don't want to develop modules and
> that will always be fine.  Others see a benefit (whether small or large)
> from developing as modules, and that should also be fine and I want
> Fedora to allow that.  Allowing modules in buildroots prevents the
> conflict between packagers who make different choices (e.g. non-modular
> Eclipse can't use module-only Maven) so seems like a good compromise.

… this is not a complete solution, because it is taking the end users 
entirely out of the equation. Allowing modules in buildroots will have no 
impact on the end user.

The net result of this proposed Change for the end user is still the same as 
the status quo: They have to use modules whether they want to or not, the 
choice is taken away from them. And while the default stream approach tries 
to hide Modularity from the users (and with this proposed Change, also from 
the packagers of dependent packages), the abstraction is leaky, as 
evidenced, e.g., by the libgit2 upgrade blocker.

I think it is not fair to force modules onto all users of the distribution 
just because it is the technical preference of a few individual developers. 
Because, in the end, it is the developers who choose to make their packages 
module-only who are forcing their way onto all users (and other developers, 
too). My proposal would only "force" the developers to give the choice back 
to the users. So I do not see myself as being the one forcing their way onto 
others. I am sorry if my suboptimal wording gave you that unfortunate 
impression.

If the user wishes to use a non-default version of a package, sure, 
Modularity can help them. But otherwise, modules should not be a requirement 
to use the distribution. There is no technical necessity for that.

Requiring packages to have a non-modular version in the non-modular 
repository does in no way preclude providing alternate versions in modules, 
so I do not see how that proposal would impede developing modules if the 
maintainer wishes to do that.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Miro Hrončok]

On 10. 10. 19 0:46, Miro Hrončok wrote:> What I miss in the description is:


1. How does this thing actually work? is there an additional repository composed 
from the default streams available in Koji only?


2. How are conflicts between packages from the default streams and ursine 
package be handled?


3. What is the local experience if the packager is using mock. What if they are 
using rpmbuild directly?


4. How are we handling default streams with dependencies on non-default streams 
of other modules?


I realize that the discussion got side tracked because of my other higher level 
concerns, but I'd still like to know about this.


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Miro Hrončok]

On 14. 10. 19 15:44, Joe Orton wrote:

I find myself a bit reluctant to write this mail because the language
others are using in this thread is fairly ugly for a technical
discussion in an open source project - about "forcing" people to develop
packages in a certain way, "teaching them a lesson" etc.  Please calm it
down & have some respect for technical decisions of other developers.


I completely agree with this message.

I don't agree with where modularity is going in Fedora, but it doesn't mean I 
don't like the people involved ;)


On the technical side, iff we decide to ban default modular stream, I don't 
think we should force people or teach them a lesson, I think we need to 
coordinate a huge effort to make everything work again - together.


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Miro Hrončok]

On 14. 10. 19 12:16, Kevin Kofler wrote:

Enable module default streams in the buildroot repository for modular
and non-modular RPMs.

== Summary ==
This Change (colloquially referred to as "Ursa Prime") enables the
Koji build-system to include the RPM artifacts provided by module
default streams in the buildroot when building non-modular (or
"traditional") RPMs.

== Owner ==
* Name: [[User:Sgallagh| Stephen Gallagher]]
* Email: sgall...@redhat.com
* Responsible WG: Modularity WG


I have already stated this multiple times in other threads, but just so that
nobody claims that I have not given feedback to this change (which has been
claimed for some other changes in a similar situation), I am going to state
this again:

I oppose this change because I believe Miro's proposal to require all
packages to have a non-modular version (i.e., to ship the default stream as
non-modular packages) is the much better approach to the issue and makes
this change proposal entirely moot.

...
* The fedora-modular repository SHOULD be disabled by default on user
   machines, matching the current build system behavior. This also helps
   enforcing the above policies.


As long as we enforce the policy of no default streams, this is actually 
unnecessary. Modules will only be used when people do "dnf module ...".


Regardless whether this is a good or a wrong idea, just makign clear that I have 
never proposed to disable modular repos by default.


OTOH I propose that disabling modular repos must be considered a legitimate 
action and the distro must remain it's core functionality.



--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Joe Orton]

On Wed, Oct 09, 2019 at 04:46:52PM -0400, Ben Cotton wrote:
> https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
> 
> Enable module default streams in the buildroot repository for modular
> and non-modular RPMs.
> 
> == Summary ==
> This Change (colloquially referred to as "Ursa Prime") enables the
> Koji build-system to include the RPM artifacts provided by module
> default streams in the buildroot when building non-modular (or
> "traditional") RPMs.
> 
> == Owner ==
> * Name: [[User:Sgallagh| Stephen Gallagher]]
> * Email: sgall...@redhat.com
> * Responsible WG: Modularity WG

Thanks for writing this up Stephen, I fully support this.

For some people here it is clear they don't want to develop modules and 
that will always be fine.  Others see a benefit (whether small or large) 
from developing as modules, and that should also be fine and I want 
Fedora to allow that.  Allowing modules in buildroots prevents the 
conflict between packagers who make different choices (e.g. non-modular 
Eclipse can't use module-only Maven) so seems like a good compromise.

I find myself a bit reluctant to write this mail because the language 
others are using in this thread is fairly ugly for a technical 
discussion in an open source project - about "forcing" people to develop 
packages in a certain way, "teaching them a lesson" etc.  Please calm it 
down & have some respect for technical decisions of other developers.

Regards, Joe
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Vitaly Zaitsev via devel]

On 14.10.2019 12:16, Kevin Kofler wrote:
> To be clear, I propose the following:
> * All packages MUST have a default version in any given Fedora release.
> * The default version MUST be shipped as non-modular (not as a modular
>   default stream).
> * It follows that packages cannot be module-only.
> * This applies to ALL packages including build tools. In particular,
>   packages must not be hidden from users through constructs such as
>   buildroot-only modules or non-API packages in modules. If a package is in
>   a module, no matter in what function, there MUST be a default version in
>   the non-modular repository.
> * The fedora-modular repository SHOULD be disabled by default on user
>   machines, matching the current build system behavior. This also helps
>   enforcing the above policies.

I totally agree with this proposals.

-- 
Sincerely,
  Vitaly Zaitsev (vit...@easycoding.org)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[John M. Harris Jr]

On Monday, October 14, 2019 3:16:27 AM MST Kevin Kofler wrote:
> > Enable module default streams in the buildroot repository for modular
> > and non-modular RPMs.
> > 
> > == Summary ==
> > This Change (colloquially referred to as "Ursa Prime") enables the
> > Koji build-system to include the RPM artifacts provided by module
> > default streams in the buildroot when building non-modular (or
> > "traditional") RPMs.
> > 
> > == Owner ==
> > * Name: [[User:Sgallagh| Stephen Gallagher]]
> > * Email: sgall...@redhat.com
> > * Responsible WG: Modularity WG
> 
> 
> I have already stated this multiple times in other threads, but just so that
>  nobody claims that I have not given feedback to this change (which has
> been claimed for some other changes in a similar situation), I am going to
> state this again:
> 
> I oppose this change because I believe Miro's proposal to require all 
> packages to have a non-modular version (i.e., to ship the default stream as
>  non-modular packages) is the much better approach to the issue and makes
> this change proposal entirely moot.
> 
> To be clear, I propose the following:
> * All packages MUST have a default version in any given Fedora release.
> * The default version MUST be shipped as non-modular (not as a modular
>   default stream).
> * It follows that packages cannot be module-only.
> * This applies to ALL packages including build tools. In particular,
>   packages must not be hidden from users through constructs such as
>   buildroot-only modules or non-API packages in modules. If a package is in
> a module, no matter in what function, there MUST be a default version in
> the non-modular repository.
> * The fedora-modular repository SHOULD be disabled by default on user
>   machines, matching the current build system behavior. This also helps
>   enforcing the above policies.
> 
> I believe that the above proposal is essentially identical to Miro's 
> proposal from the other thread, which I support. The exact wording can be 
> discussed, but it is important that the essence of the proposal (module-only
>  = no go) is retained.
> 
> Kevin Kofler
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List
> Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List
> Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

That seems to be the best approach to this situation.

-- 
John M. Harris, Jr.
Splentity

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Fabio Valentini]

On Mon, Oct 14, 2019, 12:47 Pierre-Yves Chibon  wrote:

> On Mon, Oct 14, 2019 at 12:02:46PM +0200, Kevin Kofler wrote:
> > Aleksandar Kurtakov wrote:
> > > You seem to totally miss the point - there is no one even trying to
> ship
> > > Maven as a traditional package so what should we do give up on having
> > > anything built with Maven in the distro?
> >
> > If module-only packages finally get banned (as they should have been
> from
> > the onset), the Maven maintainers will HAVE TO maintain the non-modular
> > version if they want to maintain Maven at all.
>
> You're assuming they will want to.
> I agree with Kevin Fenzi's email that the here the input from the people
> who
> have made the choice to go full-modular would be nice.
>
> If we follow your hard rule we may end up choosing between:
> - java/maven being available as module
> - no java/maven at all in Fedora
>   (this modulo the heroic efforts of the stewardship SIG - itself modulo
> the
>   fact that the SIG defines itself has a temporary solution [1])
>

FWIW, since I don't see the Java SIG rising from the ashes that were left
by it burning to the ground anytime soon, we definitely *will continue* to
maintain critical packages for the foreseeable future. So maven and all
its dependencies are safe. I also have early plans to update it to the 3.6
branch for f32.

I will later update the wiki page to reflect the current reality. The
documents in our pagure project should be up to date though.

Fabio


> You say that modularity is making people leave but this applies to any
> changes,
> including the one you are proposing here.
>
>
> Pierre
>
>
> [1] https://fedoraproject.org/wiki/SIGs/Stewardship
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Pierre-Yves Chibon]

On Mon, Oct 14, 2019 at 12:02:46PM +0200, Kevin Kofler wrote:
> Aleksandar Kurtakov wrote:
> > You seem to totally miss the point - there is no one even trying to ship
> > Maven as a traditional package so what should we do give up on having
> > anything built with Maven in the distro?
> 
> If module-only packages finally get banned (as they should have been from 
> the onset), the Maven maintainers will HAVE TO maintain the non-modular 
> version if they want to maintain Maven at all.

You're assuming they will want to.
I agree with Kevin Fenzi's email that the here the input from the people who
have made the choice to go full-modular would be nice.

If we follow your hard rule we may end up choosing between:
- java/maven being available as module
- no java/maven at all in Fedora
  (this modulo the heroic efforts of the stewardship SIG - itself modulo the
  fact that the SIG defines itself has a temporary solution [1])

You say that modularity is making people leave but this applies to any changes,
including the one you are proposing here.


Pierre


[1] https://fedoraproject.org/wiki/SIGs/Stewardship
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Pavel Raiskup]

On Thursday, October 10, 2019 5:59:16 PM CEST Miro Hrončok wrote:
> On 10. 10. 19 17:46, Igor Gnatenko wrote:
> > On Thu, Oct 10, 2019 at 12:52 AM Miro Hrončok  wrote:
> >>
> >> On 09. 10. 19 22:46, Ben Cotton wrote:
> >>> https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
> >>>
> >>> Enable module default streams in the buildroot repository for modular
> >>> and non-modular RPMs.
> >>>
> >>> == Summary ==
> >>> This Change (colloquially referred to as "Ursa Prime") enables the
> >>> Koji build-system to include the RPM artifacts provided by module
> >>> default streams in the buildroot when building non-modular (or
> >>> "traditional") RPMs.
> >>>
> >>> == Owner ==
> >>> * Name: [[User:Sgallagh| Stephen Gallagher]]
> >>> * Email: sgall...@redhat.com
> >>> * Responsible WG: Modularity WG
> >>>
> >>> == Detailed Description ==
> >>> As a major part of the Modularity design, we have a concept of default
> >>> module streams. These streams are built as modules, but the RPM
> >>> artifacts they deliver are intended to be used just like non-modular
> >>> RPMS. The aspirational goal is that a user of the system who never
> >>> executes a module-specific command (such as `dnf module install
> >>> nodejs:8`) should experience no meaningful changes in behavior from
> >>> how they would interact with a completely non-modular system. In
> >>> practice, this may mean that the informational output of package
> >>> managers may indicate that modules are being enabled and used, but a
> >>> user that does not have a specific reason to interact with the
> >>> selection of a module stream should have that managed on their behalf.
> >>
> >> If this is the goal of default modular streams, wouldn't it be in fact much
> >> easier to keep the default versions as urisne packages?
> > 
> > That means, you have 2 different workflows: for default version and
> > for an additional one.
> > When branching happens, instead of just updating one file which points
> > to the new default, you would have to create new stream, retire the
> > one which is about to become default and update package in
> > non-modular.
> 
> 
> Yes. It puts additional (arguably fairly trivial) work for the modular 
> maintainers. Other than that, it makes lives of every other maiantainar, of 
> the 
> dnf maintainers and of Fedora users easier.
> 
> I call it a good deal.

+1, that's pattern we decided to do in case of PostgreSQL.

Pavel


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Kevin Kofler]

> Enable module default streams in the buildroot repository for modular
> and non-modular RPMs.
> 
> == Summary ==
> This Change (colloquially referred to as "Ursa Prime") enables the
> Koji build-system to include the RPM artifacts provided by module
> default streams in the buildroot when building non-modular (or
> "traditional") RPMs.
> 
> == Owner ==
> * Name: [[User:Sgallagh| Stephen Gallagher]]
> * Email: sgall...@redhat.com
> * Responsible WG: Modularity WG

I have already stated this multiple times in other threads, but just so that 
nobody claims that I have not given feedback to this change (which has been 
claimed for some other changes in a similar situation), I am going to state 
this again:

I oppose this change because I believe Miro's proposal to require all 
packages to have a non-modular version (i.e., to ship the default stream as 
non-modular packages) is the much better approach to the issue and makes 
this change proposal entirely moot.

To be clear, I propose the following:
* All packages MUST have a default version in any given Fedora release.
* The default version MUST be shipped as non-modular (not as a modular
  default stream).
* It follows that packages cannot be module-only.
* This applies to ALL packages including build tools. In particular,
  packages must not be hidden from users through constructs such as
  buildroot-only modules or non-API packages in modules. If a package is in
  a module, no matter in what function, there MUST be a default version in
  the non-modular repository.
* The fedora-modular repository SHOULD be disabled by default on user
  machines, matching the current build system behavior. This also helps
  enforcing the above policies.

I believe that the above proposal is essentially identical to Miro's 
proposal from the other thread, which I support. The exact wording can be 
discussed, but it is important that the essence of the proposal (module-only 
= no go) is retained.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Kevin Kofler]

Aleksandar Kurtakov wrote:
> You seem to totally miss the point - there is no one even trying to ship
> Maven as a traditional package so what should we do give up on having
> anything built with Maven in the distro?

If module-only packages finally get banned (as they should have been from 
the onset), the Maven maintainers will HAVE TO maintain the non-modular 
version if they want to maintain Maven at all.

It is just not acceptable to move packages as central as Maven to module-
only and leave everything depending on them in the cold. This has been 
pointed out (by me and others) to the Maven maintainers ever since they 
first announced their plan, and was blatantly ignored. So we really need a 
policy to enforce social common sense (i.e., module-only = no go).

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Aleksandar Kurtakov]

On Mon, Oct 14, 2019 at 12:06 PM Fabio Valentini 
wrote:

> On Mon, Oct 14, 2019 at 10:43 AM Aleksandar Kurtakov
>  wrote:
> >
> >
> >
> > On Mon, Oct 14, 2019 at 10:13 AM John M. Harris Jr 
> wrote:
> >>
> >> On Sunday, October 13, 2019 11:42:41 PM MST Aleksandar Kurtakov wrote:
> >> > On Mon, Oct 14, 2019 at 9:00 AM John M. Harris Jr <
> joh...@splentity.com>
> >> >
> >> > wrote:
> >> > > On Wednesday, October 9, 2019 1:46:52 PM MST Ben Cotton wrote:
> >> > > >
> https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
> >> > > >
> >> > > > Enable module default streams in the buildroot repository for
> modular
> >> > > > and non-modular RPMs.
> >> > > >
> >> > > > == Summary ==
> >> > > > This Change (colloquially referred to as "Ursa Prime") enables the
> >> > > > Koji build-system to include the RPM artifacts provided by module
> >> > > > default streams in the buildroot when building non-modular (or
> >> > > > "traditional") RPMs.
> >> > > >
> >> > > > == Owner ==
> >> > > > * Name: [[User:Sgallagh| Stephen Gallagher]]
> >> > > > * Email: sgall...@redhat.com
> >> > > > * Responsible WG: Modularity WG
> >> > > >
> >> > > > == Detailed Description ==
> >> > > > As a major part of the Modularity design, we have a concept of
> default
> >> > > > module streams. These streams are built as modules, but the RPM
> >> > > > artifacts they deliver are intended to be used just like
> non-modular
> >> > > > RPMS. The aspirational goal is that a user of the system who never
> >> > > > executes a module-specific command (such as `dnf module install
> >> > > > nodejs:8`) should experience no meaningful changes in behavior
> from
> >> > > > how they would interact with a completely non-modular system. In
> >> > > > practice, this may mean that the informational output of package
> >> > > > managers may indicate that modules are being enabled and used,
> but a
> >> > > > user that does not have a specific reason to interact with the
> >> > > > selection of a module stream should have that managed on their
> behalf.
> >> > > >
> >> > > > Similarly, the experience for package maintainers of non-modular
> >> > > > packages should be unaffected by an RPM dependency moving from the
> >> > > > non-modular repository into a default module stream. Up to the
> >> > > > present, this has not been the case; no module stream content has
> been
> >> > > > available in the non-modular buildroot for other packages to
> consume.
> >> > > > Koji builds of non-modular RPMs have had only the other
> non-modular
> >> > > > RPMs from that release available to their buildroots. In contrast,
> >> > > > building on local systems has access to both the non-modular RPMs
> and
> >> > > > the RPMs from any of the default module streams. With this Change,
> >> > > > Koji builds will have the same behavior and be able to depend on
> >> > > > content provided by default module streams. It also enables the
> same
> >> > > > behavior for Modular builds: the `platform` stream will now
> include
> >> > > > the contents of the default module streams for each release and
> do not
> >> > > > need to be explicitly specified in the modulemd `buildrequires`.
> >> > > >
> >> > > > Note: This Change does not address the other major Modularity
> issue we
> >> > > > are facing around distribution upgrades with differing default
> >> > > > streams. When discussing this Change, please keep that topic
> separate.
> >> > > >
> >> > > > == Benefit to Fedora ==
> >> > > >
> >> > > > This will simplify the lives of package maintainers in Fedora in
> two
> >> > > > primary ways. I'll use a hypothetical example of the Node.js
> >> > > > interpreter and a JSApp package which is capable of running on
> Node.js
> >> > > > 10 or 12 (but requires newer features than are provided by
> Node.js 8).
> >> > > > Additionally, the JSApp package requires the same versions of
> Node.js
> >> > > > at build-time.
> >> > > >
> >> > > > * Fedora 29 ships `nodejs:8`, `nodejs:10` and `nodejs:12` module
> >> > > > streams. The `nodejs:10` stream is set as the default stream.
> >> > > > * Fedora 30 ships `nodejs:8`, `nodejs:10` and `nodejs:12` module
> >> > > > streams. The `nodejs:10` stream is set as the default stream.
> >> > > > * Fedora 31 ships `nodejs:10` and `nodejs:12` module streams. The
> >> > > > `nodejs:12` stream is set as the default stream. The `nodejs:14`
> >> > > > stream will likely become available during the F31 lifetime.
> >> > > > * Fedora 32 ships `nodejs:10` and `nodejs:12` module streams. The
> >> > > > `nodejs:12` stream is set as the default stream. The `nodejs:14`
> >> > > > stream will likely become available during the F32 lifetime.
> >> > > >
> >> > > > On Fedora 29 through 31, the Node.js package maintainer needs to
> build
> >> > > > the `nodejs:10` package both as a module and as a non-modular RPM
> in
> >> > > > the distribution so that the JSApp package can be built. With this
> >> > > > Change, the Node.js package maintainer in Fedora 32+ 

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Fabio Valentini]

On Mon, Oct 14, 2019 at 10:43 AM Aleksandar Kurtakov
 wrote:
>
>
>
> On Mon, Oct 14, 2019 at 10:13 AM John M. Harris Jr  
> wrote:
>>
>> On Sunday, October 13, 2019 11:42:41 PM MST Aleksandar Kurtakov wrote:
>> > On Mon, Oct 14, 2019 at 9:00 AM John M. Harris Jr 
>> >
>> > wrote:
>> > > On Wednesday, October 9, 2019 1:46:52 PM MST Ben Cotton wrote:
>> > > > https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
>> > > >
>> > > > Enable module default streams in the buildroot repository for modular
>> > > > and non-modular RPMs.
>> > > >
>> > > > == Summary ==
>> > > > This Change (colloquially referred to as "Ursa Prime") enables the
>> > > > Koji build-system to include the RPM artifacts provided by module
>> > > > default streams in the buildroot when building non-modular (or
>> > > > "traditional") RPMs.
>> > > >
>> > > > == Owner ==
>> > > > * Name: [[User:Sgallagh| Stephen Gallagher]]
>> > > > * Email: sgall...@redhat.com
>> > > > * Responsible WG: Modularity WG
>> > > >
>> > > > == Detailed Description ==
>> > > > As a major part of the Modularity design, we have a concept of default
>> > > > module streams. These streams are built as modules, but the RPM
>> > > > artifacts they deliver are intended to be used just like non-modular
>> > > > RPMS. The aspirational goal is that a user of the system who never
>> > > > executes a module-specific command (such as `dnf module install
>> > > > nodejs:8`) should experience no meaningful changes in behavior from
>> > > > how they would interact with a completely non-modular system. In
>> > > > practice, this may mean that the informational output of package
>> > > > managers may indicate that modules are being enabled and used, but a
>> > > > user that does not have a specific reason to interact with the
>> > > > selection of a module stream should have that managed on their behalf.
>> > > >
>> > > > Similarly, the experience for package maintainers of non-modular
>> > > > packages should be unaffected by an RPM dependency moving from the
>> > > > non-modular repository into a default module stream. Up to the
>> > > > present, this has not been the case; no module stream content has been
>> > > > available in the non-modular buildroot for other packages to consume.
>> > > > Koji builds of non-modular RPMs have had only the other non-modular
>> > > > RPMs from that release available to their buildroots. In contrast,
>> > > > building on local systems has access to both the non-modular RPMs and
>> > > > the RPMs from any of the default module streams. With this Change,
>> > > > Koji builds will have the same behavior and be able to depend on
>> > > > content provided by default module streams. It also enables the same
>> > > > behavior for Modular builds: the `platform` stream will now include
>> > > > the contents of the default module streams for each release and do not
>> > > > need to be explicitly specified in the modulemd `buildrequires`.
>> > > >
>> > > > Note: This Change does not address the other major Modularity issue we
>> > > > are facing around distribution upgrades with differing default
>> > > > streams. When discussing this Change, please keep that topic separate.
>> > > >
>> > > > == Benefit to Fedora ==
>> > > >
>> > > > This will simplify the lives of package maintainers in Fedora in two
>> > > > primary ways. I'll use a hypothetical example of the Node.js
>> > > > interpreter and a JSApp package which is capable of running on Node.js
>> > > > 10 or 12 (but requires newer features than are provided by Node.js 8).
>> > > > Additionally, the JSApp package requires the same versions of Node.js
>> > > > at build-time.
>> > > >
>> > > > * Fedora 29 ships `nodejs:8`, `nodejs:10` and `nodejs:12` module
>> > > > streams. The `nodejs:10` stream is set as the default stream.
>> > > > * Fedora 30 ships `nodejs:8`, `nodejs:10` and `nodejs:12` module
>> > > > streams. The `nodejs:10` stream is set as the default stream.
>> > > > * Fedora 31 ships `nodejs:10` and `nodejs:12` module streams. The
>> > > > `nodejs:12` stream is set as the default stream. The `nodejs:14`
>> > > > stream will likely become available during the F31 lifetime.
>> > > > * Fedora 32 ships `nodejs:10` and `nodejs:12` module streams. The
>> > > > `nodejs:12` stream is set as the default stream. The `nodejs:14`
>> > > > stream will likely become available during the F32 lifetime.
>> > > >
>> > > > On Fedora 29 through 31, the Node.js package maintainer needs to build
>> > > > the `nodejs:10` package both as a module and as a non-modular RPM in
>> > > > the distribution so that the JSApp package can be built. With this
>> > > > Change, the Node.js package maintainer in Fedora 32+ will only need to
>> > > > build the various Node.js streams and make one of them the default
>> > > > stream. The packages from it will then be added to the buildroot for
>> > > > non-modular packages. This will also make the packaging process
>> > > > somewhat more efficient, as the 

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Aleksandar Kurtakov]

On Mon, Oct 14, 2019 at 10:13 AM John M. Harris Jr 
wrote:

> On Sunday, October 13, 2019 11:42:41 PM MST Aleksandar Kurtakov wrote:
> > On Mon, Oct 14, 2019 at 9:00 AM John M. Harris Jr 
> >
> > wrote:
> > > On Wednesday, October 9, 2019 1:46:52 PM MST Ben Cotton wrote:
> > > >
> https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
> > > >
> > > > Enable module default streams in the buildroot repository for modular
> > > > and non-modular RPMs.
> > > >
> > > > == Summary ==
> > > > This Change (colloquially referred to as "Ursa Prime") enables the
> > > > Koji build-system to include the RPM artifacts provided by module
> > > > default streams in the buildroot when building non-modular (or
> > > > "traditional") RPMs.
> > > >
> > > > == Owner ==
> > > > * Name: [[User:Sgallagh| Stephen Gallagher]]
> > > > * Email: sgall...@redhat.com
> > > > * Responsible WG: Modularity WG
> > > >
> > > > == Detailed Description ==
> > > > As a major part of the Modularity design, we have a concept of
> default
> > > > module streams. These streams are built as modules, but the RPM
> > > > artifacts they deliver are intended to be used just like non-modular
> > > > RPMS. The aspirational goal is that a user of the system who never
> > > > executes a module-specific command (such as `dnf module install
> > > > nodejs:8`) should experience no meaningful changes in behavior from
> > > > how they would interact with a completely non-modular system. In
> > > > practice, this may mean that the informational output of package
> > > > managers may indicate that modules are being enabled and used, but a
> > > > user that does not have a specific reason to interact with the
> > > > selection of a module stream should have that managed on their
> behalf.
> > > >
> > > > Similarly, the experience for package maintainers of non-modular
> > > > packages should be unaffected by an RPM dependency moving from the
> > > > non-modular repository into a default module stream. Up to the
> > > > present, this has not been the case; no module stream content has
> been
> > > > available in the non-modular buildroot for other packages to consume.
> > > > Koji builds of non-modular RPMs have had only the other non-modular
> > > > RPMs from that release available to their buildroots. In contrast,
> > > > building on local systems has access to both the non-modular RPMs and
> > > > the RPMs from any of the default module streams. With this Change,
> > > > Koji builds will have the same behavior and be able to depend on
> > > > content provided by default module streams. It also enables the same
> > > > behavior for Modular builds: the `platform` stream will now include
> > > > the contents of the default module streams for each release and do
> not
> > > > need to be explicitly specified in the modulemd `buildrequires`.
> > > >
> > > > Note: This Change does not address the other major Modularity issue
> we
> > > > are facing around distribution upgrades with differing default
> > > > streams. When discussing this Change, please keep that topic
> separate.
> > > >
> > > > == Benefit to Fedora ==
> > > >
> > > > This will simplify the lives of package maintainers in Fedora in two
> > > > primary ways. I'll use a hypothetical example of the Node.js
> > > > interpreter and a JSApp package which is capable of running on
> Node.js
> > > > 10 or 12 (but requires newer features than are provided by Node.js
> 8).
> > > > Additionally, the JSApp package requires the same versions of Node.js
> > > > at build-time.
> > > >
> > > > * Fedora 29 ships `nodejs:8`, `nodejs:10` and `nodejs:12` module
> > > > streams. The `nodejs:10` stream is set as the default stream.
> > > > * Fedora 30 ships `nodejs:8`, `nodejs:10` and `nodejs:12` module
> > > > streams. The `nodejs:10` stream is set as the default stream.
> > > > * Fedora 31 ships `nodejs:10` and `nodejs:12` module streams. The
> > > > `nodejs:12` stream is set as the default stream. The `nodejs:14`
> > > > stream will likely become available during the F31 lifetime.
> > > > * Fedora 32 ships `nodejs:10` and `nodejs:12` module streams. The
> > > > `nodejs:12` stream is set as the default stream. The `nodejs:14`
> > > > stream will likely become available during the F32 lifetime.
> > > >
> > > > On Fedora 29 through 31, the Node.js package maintainer needs to
> build
> > > > the `nodejs:10` package both as a module and as a non-modular RPM in
> > > > the distribution so that the JSApp package can be built. With this
> > > > Change, the Node.js package maintainer in Fedora 32+ will only need
> to
> > > > build the various Node.js streams and make one of them the default
> > > > stream. The packages from it will then be added to the buildroot for
> > > > non-modular packages. This will also make the packaging process
> > > > somewhat more efficient, as the maintainer needs only to manage the
> > > > module stream and the MBS will build it for all configured platforms.
> > > >
> > > > Similarly, 

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[John M. Harris Jr]

On Sunday, October 13, 2019 11:42:41 PM MST Aleksandar Kurtakov wrote:
> On Mon, Oct 14, 2019 at 9:00 AM John M. Harris Jr 
> 
> wrote:
> > On Wednesday, October 9, 2019 1:46:52 PM MST Ben Cotton wrote:
> > > https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
> > > 
> > > Enable module default streams in the buildroot repository for modular
> > > and non-modular RPMs.
> > > 
> > > == Summary ==
> > > This Change (colloquially referred to as "Ursa Prime") enables the
> > > Koji build-system to include the RPM artifacts provided by module
> > > default streams in the buildroot when building non-modular (or
> > > "traditional") RPMs.
> > > 
> > > == Owner ==
> > > * Name: [[User:Sgallagh| Stephen Gallagher]]
> > > * Email: sgall...@redhat.com
> > > * Responsible WG: Modularity WG
> > > 
> > > == Detailed Description ==
> > > As a major part of the Modularity design, we have a concept of default
> > > module streams. These streams are built as modules, but the RPM
> > > artifacts they deliver are intended to be used just like non-modular
> > > RPMS. The aspirational goal is that a user of the system who never
> > > executes a module-specific command (such as `dnf module install
> > > nodejs:8`) should experience no meaningful changes in behavior from
> > > how they would interact with a completely non-modular system. In
> > > practice, this may mean that the informational output of package
> > > managers may indicate that modules are being enabled and used, but a
> > > user that does not have a specific reason to interact with the
> > > selection of a module stream should have that managed on their behalf.
> > > 
> > > Similarly, the experience for package maintainers of non-modular
> > > packages should be unaffected by an RPM dependency moving from the
> > > non-modular repository into a default module stream. Up to the
> > > present, this has not been the case; no module stream content has been
> > > available in the non-modular buildroot for other packages to consume.
> > > Koji builds of non-modular RPMs have had only the other non-modular
> > > RPMs from that release available to their buildroots. In contrast,
> > > building on local systems has access to both the non-modular RPMs and
> > > the RPMs from any of the default module streams. With this Change,
> > > Koji builds will have the same behavior and be able to depend on
> > > content provided by default module streams. It also enables the same
> > > behavior for Modular builds: the `platform` stream will now include
> > > the contents of the default module streams for each release and do not
> > > need to be explicitly specified in the modulemd `buildrequires`.
> > > 
> > > Note: This Change does not address the other major Modularity issue we
> > > are facing around distribution upgrades with differing default
> > > streams. When discussing this Change, please keep that topic separate.
> > > 
> > > == Benefit to Fedora ==
> > > 
> > > This will simplify the lives of package maintainers in Fedora in two
> > > primary ways. I'll use a hypothetical example of the Node.js
> > > interpreter and a JSApp package which is capable of running on Node.js
> > > 10 or 12 (but requires newer features than are provided by Node.js 8).
> > > Additionally, the JSApp package requires the same versions of Node.js
> > > at build-time.
> > > 
> > > * Fedora 29 ships `nodejs:8`, `nodejs:10` and `nodejs:12` module
> > > streams. The `nodejs:10` stream is set as the default stream.
> > > * Fedora 30 ships `nodejs:8`, `nodejs:10` and `nodejs:12` module
> > > streams. The `nodejs:10` stream is set as the default stream.
> > > * Fedora 31 ships `nodejs:10` and `nodejs:12` module streams. The
> > > `nodejs:12` stream is set as the default stream. The `nodejs:14`
> > > stream will likely become available during the F31 lifetime.
> > > * Fedora 32 ships `nodejs:10` and `nodejs:12` module streams. The
> > > `nodejs:12` stream is set as the default stream. The `nodejs:14`
> > > stream will likely become available during the F32 lifetime.
> > > 
> > > On Fedora 29 through 31, the Node.js package maintainer needs to build
> > > the `nodejs:10` package both as a module and as a non-modular RPM in
> > > the distribution so that the JSApp package can be built. With this
> > > Change, the Node.js package maintainer in Fedora 32+ will only need to
> > > build the various Node.js streams and make one of them the default
> > > stream. The packages from it will then be added to the buildroot for
> > > non-modular packages. This will also make the packaging process
> > > somewhat more efficient, as the maintainer needs only to manage the
> > > module stream and the MBS will build it for all configured platforms.
> > > 
> > > Similarly, from the perspective of dependent maintainers, there will
> > > no longer be anxiety about needing to move their package to a module
> > > if one or more of their dependencies drops their non-modular version
> > > in favor of a default 

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[Aleksandar Kurtakov]

On Mon, Oct 14, 2019 at 9:00 AM John M. Harris Jr 
wrote:

> On Wednesday, October 9, 2019 1:46:52 PM MST Ben Cotton wrote:
> > https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
> >
> > Enable module default streams in the buildroot repository for modular
> > and non-modular RPMs.
> >
> > == Summary ==
> > This Change (colloquially referred to as "Ursa Prime") enables the
> > Koji build-system to include the RPM artifacts provided by module
> > default streams in the buildroot when building non-modular (or
> > "traditional") RPMs.
> >
> > == Owner ==
> > * Name: [[User:Sgallagh| Stephen Gallagher]]
> > * Email: sgall...@redhat.com
> > * Responsible WG: Modularity WG
> >
> > == Detailed Description ==
> > As a major part of the Modularity design, we have a concept of default
> > module streams. These streams are built as modules, but the RPM
> > artifacts they deliver are intended to be used just like non-modular
> > RPMS. The aspirational goal is that a user of the system who never
> > executes a module-specific command (such as `dnf module install
> > nodejs:8`) should experience no meaningful changes in behavior from
> > how they would interact with a completely non-modular system. In
> > practice, this may mean that the informational output of package
> > managers may indicate that modules are being enabled and used, but a
> > user that does not have a specific reason to interact with the
> > selection of a module stream should have that managed on their behalf.
> >
> > Similarly, the experience for package maintainers of non-modular
> > packages should be unaffected by an RPM dependency moving from the
> > non-modular repository into a default module stream. Up to the
> > present, this has not been the case; no module stream content has been
> > available in the non-modular buildroot for other packages to consume.
> > Koji builds of non-modular RPMs have had only the other non-modular
> > RPMs from that release available to their buildroots. In contrast,
> > building on local systems has access to both the non-modular RPMs and
> > the RPMs from any of the default module streams. With this Change,
> > Koji builds will have the same behavior and be able to depend on
> > content provided by default module streams. It also enables the same
> > behavior for Modular builds: the `platform` stream will now include
> > the contents of the default module streams for each release and do not
> > need to be explicitly specified in the modulemd `buildrequires`.
> >
> > Note: This Change does not address the other major Modularity issue we
> > are facing around distribution upgrades with differing default
> > streams. When discussing this Change, please keep that topic separate.
> >
> > == Benefit to Fedora ==
> >
> > This will simplify the lives of package maintainers in Fedora in two
> > primary ways. I'll use a hypothetical example of the Node.js
> > interpreter and a JSApp package which is capable of running on Node.js
> > 10 or 12 (but requires newer features than are provided by Node.js 8).
> > Additionally, the JSApp package requires the same versions of Node.js
> > at build-time.
> >
> > * Fedora 29 ships `nodejs:8`, `nodejs:10` and `nodejs:12` module
> > streams. The `nodejs:10` stream is set as the default stream.
> > * Fedora 30 ships `nodejs:8`, `nodejs:10` and `nodejs:12` module
> > streams. The `nodejs:10` stream is set as the default stream.
> > * Fedora 31 ships `nodejs:10` and `nodejs:12` module streams. The
> > `nodejs:12` stream is set as the default stream. The `nodejs:14`
> > stream will likely become available during the F31 lifetime.
> > * Fedora 32 ships `nodejs:10` and `nodejs:12` module streams. The
> > `nodejs:12` stream is set as the default stream. The `nodejs:14`
> > stream will likely become available during the F32 lifetime.
> >
> > On Fedora 29 through 31, the Node.js package maintainer needs to build
> > the `nodejs:10` package both as a module and as a non-modular RPM in
> > the distribution so that the JSApp package can be built. With this
> > Change, the Node.js package maintainer in Fedora 32+ will only need to
> > build the various Node.js streams and make one of them the default
> > stream. The packages from it will then be added to the buildroot for
> > non-modular packages. This will also make the packaging process
> > somewhat more efficient, as the maintainer needs only to manage the
> > module stream and the MBS will build it for all configured platforms.
> >
> > Similarly, from the perspective of dependent maintainers, there will
> > no longer be anxiety about needing to move their package to a module
> > if one or more of their dependencies drops their non-modular version
> > in favor of a default stream. Their builds will continue to work as
> > they do today.
> >
> > == Scope ==
> > * Proposal owners:
> > # Update Packaging Guidelines with
> > [https://pagure.io/modularity/issue/146#comment-600328 requirements]
> > for module default streams
> > 

Re: Fedora 32 System-Wide Change proposal: Modules in Non-Modular Buildroot

[John M. Harris Jr]

On Wednesday, October 9, 2019 1:46:52 PM MST Ben Cotton wrote:
> https://fedoraproject.org/wiki/Changes/Modules_In_Non-Modular_Buildroot
> 
> Enable module default streams in the buildroot repository for modular
> and non-modular RPMs.
> 
> == Summary ==
> This Change (colloquially referred to as "Ursa Prime") enables the
> Koji build-system to include the RPM artifacts provided by module
> default streams in the buildroot when building non-modular (or
> "traditional") RPMs.
> 
> == Owner ==
> * Name: [[User:Sgallagh| Stephen Gallagher]]
> * Email: sgall...@redhat.com
> * Responsible WG: Modularity WG
> 
> == Detailed Description ==
> As a major part of the Modularity design, we have a concept of default
> module streams. These streams are built as modules, but the RPM
> artifacts they deliver are intended to be used just like non-modular
> RPMS. The aspirational goal is that a user of the system who never
> executes a module-specific command (such as `dnf module install
> nodejs:8`) should experience no meaningful changes in behavior from
> how they would interact with a completely non-modular system. In
> practice, this may mean that the informational output of package
> managers may indicate that modules are being enabled and used, but a
> user that does not have a specific reason to interact with the
> selection of a module stream should have that managed on their behalf.
> 
> Similarly, the experience for package maintainers of non-modular
> packages should be unaffected by an RPM dependency moving from the
> non-modular repository into a default module stream. Up to the
> present, this has not been the case; no module stream content has been
> available in the non-modular buildroot for other packages to consume.
> Koji builds of non-modular RPMs have had only the other non-modular
> RPMs from that release available to their buildroots. In contrast,
> building on local systems has access to both the non-modular RPMs and
> the RPMs from any of the default module streams. With this Change,
> Koji builds will have the same behavior and be able to depend on
> content provided by default module streams. It also enables the same
> behavior for Modular builds: the `platform` stream will now include
> the contents of the default module streams for each release and do not
> need to be explicitly specified in the modulemd `buildrequires`.
> 
> Note: This Change does not address the other major Modularity issue we
> are facing around distribution upgrades with differing default
> streams. When discussing this Change, please keep that topic separate.
> 
> == Benefit to Fedora ==
> 
> This will simplify the lives of package maintainers in Fedora in two
> primary ways. I'll use a hypothetical example of the Node.js
> interpreter and a JSApp package which is capable of running on Node.js
> 10 or 12 (but requires newer features than are provided by Node.js 8).
> Additionally, the JSApp package requires the same versions of Node.js
> at build-time.
> 
> * Fedora 29 ships `nodejs:8`, `nodejs:10` and `nodejs:12` module
> streams. The `nodejs:10` stream is set as the default stream.
> * Fedora 30 ships `nodejs:8`, `nodejs:10` and `nodejs:12` module
> streams. The `nodejs:10` stream is set as the default stream.
> * Fedora 31 ships `nodejs:10` and `nodejs:12` module streams. The
> `nodejs:12` stream is set as the default stream. The `nodejs:14`
> stream will likely become available during the F31 lifetime.
> * Fedora 32 ships `nodejs:10` and `nodejs:12` module streams. The
> `nodejs:12` stream is set as the default stream. The `nodejs:14`
> stream will likely become available during the F32 lifetime.
> 
> On Fedora 29 through 31, the Node.js package maintainer needs to build
> the `nodejs:10` package both as a module and as a non-modular RPM in
> the distribution so that the JSApp package can be built. With this
> Change, the Node.js package maintainer in Fedora 32+ will only need to
> build the various Node.js streams and make one of them the default
> stream. The packages from it will then be added to the buildroot for
> non-modular packages. This will also make the packaging process
> somewhat more efficient, as the maintainer needs only to manage the
> module stream and the MBS will build it for all configured platforms.
> 
> Similarly, from the perspective of dependent maintainers, there will
> no longer be anxiety about needing to move their package to a module
> if one or more of their dependencies drops their non-modular version
> in favor of a default stream. Their builds will continue to work as
> they do today.
> 
> == Scope ==
> * Proposal owners:
> # Update Packaging Guidelines with
> [https://pagure.io/modularity/issue/146#comment-600328 requirements]
> for module default streams
> # Create a Pungi configuration to generate the buildroot from the
> default module streams.
> # Include `default_modules_scm_url` in the platform virtual module
> specification
 # Configure Koji tags for inheriting the new
>