Re: [Fedora-packaging] Schedule for Wednesday's FPC Meeting (2017-08-09 17:00 UTC)
On Wed, 2017-08-09 at 01:28 -0400, James Antill wrote: > Following is the list of topics that will be discussed in the FPC > meeting Thursday at 2017-08-09 17:00 UTC in #fedora-meeting-3 on > irc.freenode.net. Note that this should be #fedora-meeting-2 as that is free. > Local time information (via. uitime): > > = Day: Wednesday = > 2017-08-09 10:00 PDT US/Pacific > 2017-08-09 13:00 EDT --> US/Eastern <-- > 2017-08-09 17:00 UTC UTC > 2017-08-09 18:00 BST Europe/London > 2017-08-09 19:00 CEST Europe/Berlin > 2017-08-09 19:00 CEST Europe/Paris > 2017-08-09 22:30 IST Asia/Calcutta > --- New Day: Thursday > 2017-08-10 01:00 HKT Asia/Hong_Kong > 2017-08-10 01:00 +08 Asia/Singapore > 2017-08-10 02:00 JST Asia/Tokyo > 2017-08-10 03:00 AEST Australia/Brisbane > > Links to all tickets below can be found at: > > https://pagure.io/packaging-committee/issues?status=Open&tags=meeting > > = Followups = > > #topic #654 glibc file triggers > .fpc 654 > https://pagure.io/packaging-committee/issue/654 > > #topic #691 noarch *sub*packages with arch-specific dependencies > .fpc 691 > https://pagure.io/packaging-committee/issue/691 > > #topic #693 Wiki:Packaging:RPMMacros > .fpc 693 > https://pagure.io/packaging-committee/issue/693 > > #topic #696 Packages not following the Guidelines for bundled libraries > .fpc 696 > https://pagure.io/packaging-committee/issue/696 > > = New business = > > #topic #697 simplify github urls > .fpc 697 > https://pagure.io/packaging-committee/issue/697 > > #topic #698 Forbid the use of /usr/bin/python > .fpc 698 > https://pagure.io/packaging-committee/issue/698 > > #topic #700 Globally ban use of /usr/bin/env in executables > .fpc 700 > https://pagure.io/packaging-committee/issue/700 Also adding: #topic #705 Rust Packaging Guidelines .fpc 705 https://pagure.io/packaging-committee/issue/705 > = Open Floor = > > For more complete details, please visit each individual ticket. The > report of the agenda items can be found at: > > https://pagure.io/packaging-committee/issues?status=Open&tags=meeting > > If you would like to add something to this agenda, you can reply to > this e-mail, file a new ticket at https://pagure.io/packaging-committee, > e-mail me directly, or bring it up at the end of the meeting, during > the open floor topic. Note that added topics may be deferred until > the following meeting. > > ___ > packaging mailing list -- packag...@lists.fedoraproject.org > To unsubscribe send an email to packaging-le...@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Schedule for Wednesday's FPC Meeting (2017-08-09 17:00 UTC)
Following is the list of topics that will be discussed in the FPC meeting Thursday at 2017-08-09 17:00 UTC in #fedora-meeting-3 on irc.freenode.net. Local time information (via. uitime): = Day: Wednesday = 2017-08-09 10:00 PDT US/Pacific 2017-08-09 13:00 EDT --> US/Eastern <-- 2017-08-09 17:00 UTC UTC 2017-08-09 18:00 BST Europe/London 2017-08-09 19:00 CEST Europe/Berlin 2017-08-09 19:00 CEST Europe/Paris 2017-08-09 22:30 IST Asia/Calcutta --- New Day: Thursday 2017-08-10 01:00 HKT Asia/Hong_Kong 2017-08-10 01:00 +08 Asia/Singapore 2017-08-10 02:00 JST Asia/Tokyo 2017-08-10 03:00 AEST Australia/Brisbane Links to all tickets below can be found at: https://pagure.io/packaging-committee/issues?status=Open&tags=meeting = Followups = #topic #654 glibc file triggers .fpc 654 https://pagure.io/packaging-committee/issue/654 #topic #691 noarch *sub*packages with arch-specific dependencies .fpc 691 https://pagure.io/packaging-committee/issue/691 #topic #693 Wiki:Packaging:RPMMacros .fpc 693 https://pagure.io/packaging-committee/issue/693 #topic #696 Packages not following the Guidelines for bundled libraries .fpc 696 https://pagure.io/packaging-committee/issue/696 = New business = #topic #697 simplify github urls .fpc 697 https://pagure.io/packaging-committee/issue/697 #topic #698 Forbid the use of /usr/bin/python .fpc 698 https://pagure.io/packaging-committee/issue/698 #topic #700 Globally ban use of /usr/bin/env in executables .fpc 700 https://pagure.io/packaging-committee/issue/700 = Open Floor = For more complete details, please visit each individual ticket. The report of the agenda items can be found at: https://pagure.io/packaging-committee/issues?status=Open&tags=meeting If you would like to add something to this agenda, you can reply to this e-mail, file a new ticket at https://pagure.io/packaging-committee, e-mail me directly, or bring it up at the end of the meeting, during the open floor topic. Note that added topics may be deferred until the following meeting. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: 'No More Alphas': wiki revision drafts
On Tue, Aug 08, 2017 at 04:24:16PM -0700, Adam Williamson wrote: > > Is there any chance of running that at, say, > > https://qa.fedoraproject.org/blockerbugs-nextrelease/ instead of a > > local instance? > I mean, in *theory* we could for sure, it's just yet another > maintenance task for someone, and I'm not jumping up to volunteer for > it right now...if we could script the update of the instances a bit > more (right now one of us has to go into the admin UI and update it > manually every milestone), maybe I'd be more inclined :P Is the maintenance headache in installing and running/updating the service, or is it in this milestone updating? If it's the latter, we could probably ask Jan to handle it as part of FPgM duties. Would that help? (Probably for both the "nextrelease" *and* current app.) -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Self Introduction: David J Battaglia
Hello and thank you. What is the best direction to get started with doing reviews and or helping out triaging/fixing bugs. Just find something in bugzilla to work on ? Regards On Tue, Aug 8, 2017 at 10:39 PM, Zbigniew Jędrzejewski-Szmek < zbys...@in.waw.pl> wrote: > On Tue, Aug 08, 2017 at 09:01:59PM -0400, djb djb wrote: > > Hello, > > > > I am just starting out with fedora packaging. My background is mostly > > working in NOCs, infrastructure support roles and working as sys admin. > > > > Look for forward to learning as much as possible with the hopes of > getting > > sponsored. > > Hi, > > welcome to Fedora. > > Do you want to submit some packages of your own? If yes, it's good > to start with that, you should get some useful advice. Make sure to > mark the review ticket as blocking FE-NEEDSPONSOR. > > It usually helps if you do some informal reviews of other packages > or help with triaging or fixing some bugs. It's a required skill for > packagers, and a way to gain experience with the system and to show > that you know what you are doing. > > Zbyszek > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Self Introduction: David J Battaglia
On Tue, Aug 08, 2017 at 09:01:59PM -0400, djb djb wrote: > Hello, > > I am just starting out with fedora packaging. My background is mostly > working in NOCs, infrastructure support roles and working as sys admin. > > Look for forward to learning as much as possible with the hopes of getting > sponsored. Hi, welcome to Fedora. Do you want to submit some packages of your own? If yes, it's good to start with that, you should get some useful advice. Make sure to mark the review ticket as blocking FE-NEEDSPONSOR. It usually helps if you do some informal reviews of other packages or help with triaging or fixing some bugs. It's a required skill for packagers, and a way to gain experience with the system and to show that you know what you are doing. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Self Introduction: David J Battaglia
Hello, I am just starting out with fedora packaging. My background is mostly working in NOCs, infrastructure support roles and working as sys admin. Look for forward to learning as much as possible with the hopes of getting sponsored. Regards, David J Battaglia ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: 'No More Alphas': wiki revision drafts
On Fri, 2017-08-04 at 00:38 -0400, Matthew Miller wrote: > On Thu, Aug 03, 2017 at 08:50:09PM -0700, Adam Williamson wrote: > > > Will the blockerbugs app start showing F28 blockers at the F27 branch > > > point? Or sometime later? From my point of view, the sooner the better. > > > > It tends to go wrong in various ways when you have multiple releases > > 'active' at once, so we've stopped doing that lately, I'm afraid. You > > can quite easily deploy a local instance, though, and configure it to > > show the F28 blocker trackers; the bugs themselves are created two > > releases in advance, so the F28 blocker tracker bugs already exist. > > Is there any chance of running that at, say, > https://qa.fedoraproject.org/blockerbugs-nextrelease/ instead of a > local instance? I mean, in *theory* we could for sure, it's just yet another maintenance task for someone, and I'm not jumping up to volunteer for it right now...if we could script the update of the instances a bit more (right now one of us has to go into the admin UI and update it manually every milestone), maybe I'd be more inclined :P -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Mass package change (python2- binary package renaming)
Hello Fedora Python package maintainers!
This is an announcement of a mass package renaming:
Python 2 binary packages will be renamed to python2-*.
This will happen soon after the F27 branching on August 15th.
Currently ~1330 source packages already generate a binary package with
the python2- prefix, and 835 remain to be updated. The spec files for
approximately 740 packages will be renamed, and 95 will be left for
fixing by maintainers or proven packagers.
At the end of this e-mail are two lists of maintainers and packages:
List 1. for those packages which will be taken care of by the mass remaining
Patches: https://in.waw.pl/~zbyszek/fedora/pyrename/
Maintainers don't have to do anything.
List 2. for the remaining packages
Maintainers are encouraged to update packages manually.
### Details of the change ###
The change is to ensure that as many as possible python packages which
provide a version for python2 have a python2- subpackage as required by
the guidelines
[https://fedoraproject.org/wiki/Packaging:Naming#Python2_binary_package_naming].
For source packages which do *not* have a python- prefix, but already
have a python-foo subpackage, that subpackage will be renamed to python2-foo.
For packages which are called python-foo and just build a main binary package
python-foo, a new python2-foo subpackage will be created.
Provides/Requires/Conflicts/Obsoletes/Recommends/Suggests are moved from the
main package to the new subpackage.
In all cases, the new python2- subpackage will use lower-case naming.
This may lead to a situation where an existing python3- subpackage has
a differently-cased name than the python2- subpackage.
An effort is made (thanks Miro!) to preserve the style which is used
in the spec file, so that if e.g. python-%{real_name} was used originally,
this is changed to python2-%{real_name}. If this macro cannot be guessed or
if it resolves to a non-lowercase name, a non-macro lower-case string will
be used.
%{python_provide python2-foobar} are added as required by the guidelines
[https://fedoraproject.org/wiki/Packaging:Python#The_.25python_provide_macro].
One or two: %{python_provide python2-foobar} is always added, and if the package
has a non-lower-case name, %{python_provide python2-FooBar} is also added.
This provides backward compatible names to the package (python-foobar,
python-FooBar),
as long as %{python_provide} is defined as it is now. Once %python_provide macro
is switched to prefer python3 subpackages, those compat names will be gone.
In case the source package name does not have a python- prefix, a Provides
for the old name is also added, with a comment that it should be removed later.
Example:
+%package -n python2-atpy
+Summary: %summary
+Requires: numpy python-astropy
+%{?python_provide:%python_provide python2-atpy}
+# Remove before F30
+Provides: ATpy = %{version}-%{release}
Changes are implemented in an automated way using
https://pagure.io/pyrenamer.
rpmdev-bumpspec is used to add changelog entries will be added that look like:
* Sat Aug 05 2017 Zbigniew Jędrzejewski-Szmek - 3.3-4
- Python 2 binary package renamed to python2-foobar
See https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3
### Which packages need to change ###
http://fedora.portingdb.xyz/namingpolicy/ "Misnamed subpackage" lists 877
subpackages.
The real number is a bit lower, because there are some dead packages
and some that have been fixed in the meanwhile on that list.
The automated rename currently works for 737 packages, 7 have manual patches,
47 don't need work.
The remaining 95 packages will require manual fixes.
I do *not* want maintainers to manually fix packages which are covered
by the automated changes (list 1 below), since that provides no
benefit. Maintainers are of course encouraged to fix packages which
are not covered (list 2 below), or to take the automated patch and use
it as a basis of their own fix if the automated change is deficient
for some reason.
Help with the remaining ~95 packages which cannot be done automatically
is very much welcome.
### Opting out and independent fixes ###
The script skips all packages which already have a python2- subpackage
and %python_provide, so all packages which are fixed in the meantime will be
skipped. If packages need to be skipped even without that, let me know and
I'll add the package to the blacklist.
### Timeline ###
2017-08-08: this e-mail is sent to fedora-announce
soon after 2017-08-15 (F27 branching): automatically and manually
generated patches are applied, and packages are rebuilt.
Depending on the results, I'll fix some and follow up with bugzilla
tickets for the remaining packages.
### List 1. Packages handled by the automated rename ###
[https://in.waw.pl/~zbyszek/fedora/pyrename/]
Maintainers by package:
ATpy sergiopr
CheMPS2 talcite
NLoptbesser82
OpenIPMI branto jridky pknirsch
OpenImageIO hobbes1069
PyPAM
Re: Module Stream "Expansion"
On Tue, 2017-08-08 at 13:39 -0400, Ralph Bean wrote: > ## Solution: "Input" Modulemd Syntax Changes > > We’re going to extend the modulemd syntax to allow specifying multiple > dependencies in an "input" modulemd (the one that packagers modify). When > submitted to the build system, the module-build-service (MBS) will *expand* > these values under the hood, and submit **multiple** module builds to > itself based on the expansion. That sounds like a very powerful idea! I see two different possible motivations for this, one good, and one maybe not so much. The good: if you have a component like 'httpd' that's part of your application, it's pretty clear that you don't want your choice of httpd version to force you into a particular Fedora platform and a particular version of all your other components. The bad: you have a Fedora system installed with the f27 version of the Host module. You want to install an application on top of that, so you need to have the application available built on the f27 runtime. One reason I don't like the second motivation is that it feeds the thinking that modules can be arbitrarily combined within the same namespace, and they cannot. Even if two modules are built against the same runtime, the same version of Python, they still can have conflicting versions of libraries. That's basically the point of modularity. But beyond that, if maintainers think that their application module has to be built and tested against every active platform version (plus whatever other combinatorial explosion users ask for), then any simplification that modularity provides to packagers is gone, and instead people will be constantly fighting library compatibility and adding conditionals, and generally tearing their hair out. So I think we need to be very clear in the expectations that we're setting for module maintainers - that this is a new tool in the toolbox of a maintainer, not a hoop that they have to jump through. And if it makes sense to package your module as a container or a flatpak then that is an entirely satisfactory way to handle combining your module with modules built on a different platform version. Owen ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Module Stream "Expansion"
On Tue, Aug 08, 2017 at 03:11:38PM -0400, Matthew Miller wrote: > On Tue, Aug 08, 2017 at 09:04:40PM +0200, Petr Šabata wrote: > > dependencies: > > buildrequires: &deps > > platform: [f28, f27, f26] > > shared-userspace: [fancy, nonfancy] > > requires: *deps > > > > Another point that came up later -- instead of shared-userspace, > > imagine different streams of your favorite dynamic language. > > That might make it the reasons for this more obvious. > > So, like: > > dependencies: > > buildrequires: &deps > > platform: [f28, f27, f26] > > python: [2.7, 3.6] > > requires: *deps > > > Hmmm, though. This doesn't really make sense if the module in question > just happens to use python. In that case, why build it for different > dependencies? Just pick one; the user won't care. If you pick one, you make your module unavailable to people who chose to install the incompatible other on their system. Building it (automatically!) multiple times allows the user to choose, rather then the developer forcing their choice onto them. > If on the other hand, the user *does* care (perhaps the module is a > framework where more stuff is then written in the underlying language, > so the user wants to choose). In that case, wouldn't we *want* that > surfaced into the stream name? No. This allows us to ship the same build in multple variants and no matter what platform and python (following the abovementioned example) the user has installed, the right binary variant of your module will be deployed. If they decide to switch to another platform or python, your application will be switched for the variant which is compatible with those. The variants will be distinguished by the context flag, mostly in the background. P signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Module Stream "Expansion"
On Tue, Aug 08, 2017 at 09:04:40PM +0200, Petr Šabata wrote: > dependencies: > buildrequires: &deps > platform: [f28, f27, f26] > shared-userspace: [fancy, nonfancy] > requires: *deps > > Another point that came up later -- instead of shared-userspace, > imagine different streams of your favorite dynamic language. > That might make it the reasons for this more obvious. So, like: dependencies: buildrequires: &deps platform: [f28, f27, f26] python: [2.7, 3.6] requires: *deps Hmmm, though. This doesn't really make sense if the module in question just happens to use python. In that case, why build it for different dependencies? Just pick one; the user won't care. If on the other hand, the user *does* care (perhaps the module is a framework where more stuff is then written in the underlying language, so the user wants to choose). In that case, wouldn't we *want* that surfaced into the stream name? -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Module Stream "Expansion"
On Tue, Aug 08, 2017 at 01:39:45PM -0400, Ralph Bean wrote: > This is a writeup on a problem we’re facing with modularity, and some ideas on > how to resolve it. > > # The "Problem" > > Imagine I have an **httpd module**. To simplify things, let’s say that this > module has only one stream: **2.4**. Today, in the modulemd for this module, I > declare build and runtime dependencies like this: > > dependencies: > buildrequires: > platform: f26 > requires: > platform: f26 > > This works. > > Now, imagine that Fedora 27 comes along. I want the httpd module to be > installable on f27, so I update those streams to point to f27. But now I > can’t > ship updates to f26 anymore! Uh oh. > > ## Just use more streams? > > We could just ask packagers to create more streams to deal with this. httpd > wouldn’t have a 2.4 stream. It would need to have a f26-2.4 stream and a > f27-2.4 stream. > > But, this gets us exactly back into the place we were **before** we had > arbitrary branching! You need to have as many branches for *every component* > as you have releases of the distro (or, as you have "products"). N! > > ## Solution: "Input" Modulemd Syntax Changes > > We’re going to extend the modulemd syntax to allow specifying multiple > dependencies in an "input" modulemd (the one that packagers modify). When > submitted to the build system, the module-build-service (MBS) will *expand* > these values under the hood, and submit **multiple** module builds to > itself based on the expansion. > > Here are some examples of modulemd files (maintained by humans) that might be > submitted to the MBS. > > Build on **all** active streams of the platform module. > dependencies: > buildrequires: > platform: [] > > Build on **only** the f27 and f26 platform streams. > dependencies: > buildrequires: > platform: [ f27, f26 ] > > Build on all active streams of platform **except** for the f26 stream. > dependencies: > buildrequires: > platform: [ -f26 ] Maybe it should be explicitly mentioned that combining the two variants above will be forbidden. > The following syntax, which builds on **only** the f26 stream, will still be > supported: > dependencies: > buildrequires: > platform: f26 And this is not only for backwards compatibility -- it's also what the expanded variant will look like. > Take the following example (described above): > dependencies: > buildrequires: > platform: [ f27, f26 ] > > Submitting this modulemd to the MBS would in turn generate **two** module > builds: one of our httpd module against the f27 stream and another against the > f26 stream. Each module build would be associated with its own unique > "flattened" *output* modulemd file that specifies exactly which platform > stream > it was built against. > > # New Problems > > Having **multiple** module builds for a single dist-git commit of a modulemd > file poses new implementation problems. > > ## NSV uniqueness > > Today, we uniquely identify a module build in *a variety of systems* with > **--** (NSV) where version is derived from the dist-git > commit timestamp. Here we’ll have an httpd-2.4 module built on f26 and an > http-2.4 module built on f27: two different module builds with the same name, > stream, and version. How can we tell them apart? > > We will introduce a new value called the **context** of a built module and > include it in the modulemd metadata that gets carried with the built module. > > * For user facing cases (dnf installation) it will generally be hidden. The > old NSV value will still appear. If a user ever needs to surface the value, > client tooling can find it in the modulemd metadata. The additional > identifier will give users access to explicit pointers to the content that > is > installed: > * For cloning a machine. > * For comparing two installed hosts. > * For reporting bugs. > * Some build and compose time systems will have to be modified to use the > context as part of a new unique identifier. **NSVC** will be the > **---**. The systems in question are PDC, > koji/brew, pungi, and bodhi. Perhaps a side note: The exact way of encoding all the module identifiers into a string is still being discussed. I would very much prefer if we used the same format in all systems that have to deal with modules -- not only the infrastructure services listed above but dnf, for instance, as well. Some proposals and comments: https://pagure.io/modularity/pull-request/43 > The context value will be a **hash**, generating as the first step in the > build > process (but after expansion). Consider what metadata needs to be hashed: we > think that hashing the whole modulemd is problematic, because the modulemd can > and will be modified after build time. > > Therefore, the *context_hash* value will need to be de
Fedora 24 End Of Life
As of the 8th of August 2017, Fedora 24 has reached its end of life for updates and support. No further updates, including security updates, will be available for Fedora 24. A previous reminder was sent on 21st of July 2017 [0]. Fedora 25 will continue to receive updates until approximately one month after the release of Fedora 27. The maintenance schedule of Fedora releases is documented on the Fedora Project wiki [1]. The Fedora Project wiki also contains instructions [2] on how to upgrade from a previous release of Fedora to a version receiving updates. Mohan Boddu. [0]https://lists.fedoraproject.org/archives/list/devel@lists. fedoraproject.org/message/DNMTG4GZJRH6E3WLBNWVPUPGLESLOKBN/ [1]https://fedoraproject.org/wiki/Fedora_Release_Life_Cycle# Maintenance_Schedule [2]https://fedoraproject.org/wiki/Upgrading?rd=DistributionUpgrades ___ devel-announce mailing list -- devel-annou...@lists.fedoraproject.org To unsubscribe send an email to devel-announce-le...@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Module Stream "Expansion"
This is a writeup on a problem we’re facing with modularity, and some ideas on how to resolve it. # The "Problem" Imagine I have an **httpd module**. To simplify things, let’s say that this module has only one stream: **2.4**. Today, in the modulemd for this module, I declare build and runtime dependencies like this: dependencies: buildrequires: platform: f26 requires: platform: f26 This works. Now, imagine that Fedora 27 comes along. I want the httpd module to be installable on f27, so I update those streams to point to f27. But now I can’t ship updates to f26 anymore! Uh oh. ## Just use more streams? We could just ask packagers to create more streams to deal with this. httpd wouldn’t have a 2.4 stream. It would need to have a f26-2.4 stream and a f27-2.4 stream. But, this gets us exactly back into the place we were **before** we had arbitrary branching! You need to have as many branches for *every component* as you have releases of the distro (or, as you have "products"). N! ## Solution: "Input" Modulemd Syntax Changes We’re going to extend the modulemd syntax to allow specifying multiple dependencies in an "input" modulemd (the one that packagers modify). When submitted to the build system, the module-build-service (MBS) will *expand* these values under the hood, and submit **multiple** module builds to itself based on the expansion. Here are some examples of modulemd files (maintained by humans) that might be submitted to the MBS. Build on **all** active streams of the platform module. dependencies: buildrequires: platform: [] Build on **only** the f27 and f26 platform streams. dependencies: buildrequires: platform: [ f27, f26 ] Build on all active streams of platform **except** for the f26 stream. dependencies: buildrequires: platform: [ -f26 ] The following syntax, which builds on **only** the f26 stream, will still be supported: dependencies: buildrequires: platform: f26 -- Take the following example (described above): dependencies: buildrequires: platform: [ f27, f26 ] Submitting this modulemd to the MBS would in turn generate **two** module builds: one of our httpd module against the f27 stream and another against the f26 stream. Each module build would be associated with its own unique "flattened" *output* modulemd file that specifies exactly which platform stream it was built against. # New Problems Having **multiple** module builds for a single dist-git commit of a modulemd file poses new implementation problems. ## NSV uniqueness Today, we uniquely identify a module build in *a variety of systems* with **--** (NSV) where version is derived from the dist-git commit timestamp. Here we’ll have an httpd-2.4 module built on f26 and an http-2.4 module built on f27: two different module builds with the same name, stream, and version. How can we tell them apart? We will introduce a new value called the **context** of a built module and include it in the modulemd metadata that gets carried with the built module. * For user facing cases (dnf installation) it will generally be hidden. The old NSV value will still appear. If a user ever needs to surface the value, client tooling can find it in the modulemd metadata. The additional identifier will give users access to explicit pointers to the content that is installed: * For cloning a machine. * For comparing two installed hosts. * For reporting bugs. * Some build and compose time systems will have to be modified to use the context as part of a new unique identifier. **NSVC** will be the **---**. The systems in question are PDC, koji/brew, pungi, and bodhi. The context value will be a **hash**, generating as the first step in the build process (but after expansion). Consider what metadata needs to be hashed: we think that hashing the whole modulemd is problematic, because the modulemd can and will be modified after build time. Therefore, the *context_hash* value will need to be derived from only the stuff that uniquely identifies the build and runtime context of the module -- name, stream, version and crucially, its dependencies ## Buildtime/Runtime Dep Correlation Another problem. We now have input modulemd files for a single stream that can expand buildtime dependencies to ‘f27’ and ‘f26’, but what about the runtime dependencies? Consider the following yaml file: dependencies: buildrequires: platform: [f28, f27, f26] requires: platform: [f28, f27, f26] With our thinking caps on, this should obviously generate three module builds (one that buildrequires f28 and run requires f28, a second that buildrequires f27 and run requires f27, and a third for f26). The naive cross-product of all streams is not valuable; the MBS needs some logic to **correlate** the build time requirements with the run
Re: Arbitrary Branching and solving the "Change Checkpoint? Better hit the gas!" problem
On Tue, Aug 08, 2017 at 03:26:04PM +0200, Petr Šabata wrote: > > Hm. I agree entirely with you, but when reading this I thought of > > another possibility. I think modularity gives people the option for a > > rolling release, where we don't have to make release == "collection of > > specific module versions". If that's one of the outcomes, then > > Changes simply become "this module version is available". > I agree. Once everything is modular, distro-wide release changes > stop making sense. Instead we should be proposing changes for > modules -- announcements of streams and such. Well, for some things, like "GNOME updated to 3.24", definitely. Others might be policy changes we want applied to all modules (new compiler flags or security choices, say) — although for these progress could be shown as "list of modules which have the change active". > Fedora release could/should still be driven by a set of low level > modules, such as Platform and a few more tightly linked with it. Assuming we are successfully all-in on this, I'd like to have separation of what we mean by "release". First a release of Platform and tightly-linked modules twice a year. Second, releases of updated Editions and Spins built on top of that. The former would be largely an internal event, and the latter would be user (and press) focused. And we could do things like update Server once a year, Atomic every two weeks, Workstation twice a year, KDE three times a year (or whatever). > Would it make sense to split the compose into two or more in this > case? "Platform and stuff" + "Other, mostly independent stuff"? I'd like to see the compose split up as much as possible. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Arbitrary Branching and solving the "Change Checkpoint? Better hit the gas!" problem
On Mon, Aug 07, 2017 at 07:58:35PM +, Langdon White wrote: > We haven't documented this yet because we have been working through the > details of the how it should work. Basically, we need to provide a way, on > the system, to define: > a) what the "release" is. In other words, what did the Edition-WG decide > should be *installed* by default and what should be *available* by default. > For example, less version 487 should be installed, and httpd-2.4 should be > available. > b) how to "walk" the streams, hopefully automatically. In other words, if a > user makes no changes, how does s/he move from foo:1 -> foo:1.1 (where "1" > and "1.1" are different streams) vs foo:1 -> foo:2. And, in a related way, > how can s/he choose *not* to follow the guidelines. For example, I am > running a simple html website. I want to follow every upgrade to httpd that > comes out, assuming it doesn't change it's configuration method (so, auto > jump httpd2.4->httpd2.6). However, my php website should stick to > httpd2.4.z. We don't need "b" for F27, but if we build any deliverables for it using Modularity, we sure do need it for F28. Otherwise, users have no upgrade path. > We think this needs a simple DSL kinda like python requirements, nodejs > package, or even like rpm. We needed Boltron to make how this problem is > expressed "real." I am glad the question is coming up ;) "We need a new DSL" sends shivers down my spine. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [Modularity]: Service levels and EOL expectations?
On Mon, Aug 07, 2017 at 11:48:53PM +, Langdon White wrote: > I guess I am not sure how this is different with modules than with Fedora > today. We promise a 13 month lifecycle on openssl (and everything else) > already. I think the difference here is only that the "position" is > explicit. However, this is not the "real" point to my reply. Well, right now, that implicit SL/lifecycle is the same for every package -- or given a specific exception https://fedoraproject.org/wiki/Updates_Policy#Exceptions. If we add the ability to make it explicit, we need to now be clear on which choices are acceptable in Fedora. If we make it "you've got to have the exact SL and lifecycle as always before", we lose quite a bit of the power. If we make it "sure, go EOL every two weeks with your core package!" we lose the benefit of being a distro at all. > I agree it is true that the lifecycle of a given *binary* is locked to the > lifecycle of the module binaries it depends on. However. this does not > necessarily mean that a *stream* is locked to the those lifecycles. In > other words, wordpress doesn't expose glibc or openssl APIs to its > consumers (to my knowledge ;) ). As a result, it is perfectly valid for the > wordpress-4 stream to be built against base-runtime-f26 (brt) or > base-runtime-f27 or both. If the lifecycle of base-runtime-f26 runs out, > that doesn't mean wordpress-4's lifecycle has run out, it just means that > the user must upgrade the base-runtime stream but their application that > relies on the wordpress-4 API will continue to operate unchanged. Sure. But from a user point of view, if this juggling happens on arbitrary dates many times throughout the calendar year, we've just delivered something with all the disadvantages of a rolling release *plus* a lot of extra complication. But let's not even go that far out. Just within base runtime itself, there are hundreds of packages. If *those* are aribitrarily EOLing all the time, the maintainers of the base runtime module are going to have to spend significant time juggling that. As what I read this > As we are able to increase the bundling of components, the 99% continues to > add 9s after the decimal. At present, we must consume the openssl-1.1 > branch in brt/shared-userspace because dnf/rpm can't tell that the binaries > provided by different modules are functionally the same. However, as we > improve that or find other methods around this problem (private > namespacing, containers, rpathing, etc), we could have the openssl-1.1 > sources in dist-git be included in multiple modules. When the openssl > maintainer elects to add the openssl-1.2 stream, wordpress-4 could decide > if it is appropriately backwards compatible for their use case and consume > it (by changing their modulemd to point to the new stream). ... a lot of that juggling is going to be discovering every week that some important component has shifted and now the module maintainers need to find someone to maintain a compat branch of the older package. That could include packages which are just under the hood but incompatible or could be packages which have a knock-on effect and change the exposed module API. Following all of this seems like an unrealistic explosion of work. Right now, if I want to maintain Apache, I can rely on the 13-month implicit API for OpenSSL. I can concentrate on the thing I have expertise and interest in and trust that others in the community have made commitments for the dependencies. It's great that modularity offers me the ability to take on more if I want to, but if I *have* to in order to make a module with even the same promises as current Fedora, that's worrisome. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [Modularity]: Service levels and EOL expectations?
On Tue, Aug 08, 2017 at 10:38:15AM -0400, Przemek Klosowski wrote: > >Yeah, that would get crazy fast. The 6 month granularity proposal > >should help*some*, and we should probably go into this carefully. > > Technically, the SL for the module could have the narrow meaning > referring to the module only, and the tools could follow the chain > of dependencies and display it as: > > Service Level: 6 months (effective SL: 3 months/EOL: 2017-11-08 due > to dependency on openSSL) Ouch. Yes, but I think that that should be a warning sent to the module maintainers (and collectively to devel list) rather than show to end users. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [Modularity]: Service levels and EOL expectations?
On Tue, Aug 08, 2017 at 02:13:38PM -, Ralph Bean wrote: > Thanks for starting this. I'm not aware of a ticket or a responsible > party at this point. +1 to working towards formalizing this. I'll make one if no one else has. Should we start at a rel-eng ticket, get a proposal worked out, and then propose to FESCo? [...] > FYI - we had to put some initial SL values into the database to seed > the branch migration. Our existing 'master', fedora, and epel > branches needed values in the new database. Here are the ones we > added: > > - rawhide - Our thought was that the 'rawhide' SL would signal whatever > rawhide means today: more or less tracking upstream. The master branch of > all packages got this SL applied in our migration. > - bug_fixes - See security fixes below. > - security_fixes - This, along with bug_fixes, was a generic SL that got > applied to all "fedora" branches (f26, f25, etc..). > - stable_api - This one got applied to all of the epel branches. > > https://pdc.stg.fedoraproject.org/rest_api/v1/component-sla-types/ So this is a list of values? Interesting. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Modularity Working Group IRC Meeting Minutes (2017-08-08)
= #fedora-meeting-3: Meeting of the Modularity Working Group (once every two weeks) = Meeting started by nils at 14:00:21 UTC. Minutes: https://meetbot.fedoraproject.org/fedora-meeting-3/2017-08-08/modularity_wg.2017-08-08-14.00.html Minutes (text): https://meetbot.fedoraproject.org/fedora-meeting-3/2017-08-08/modularity_wg.2017-08-08-14.00.txt Log: https://meetbot.fedoraproject.org/fedora-meeting-3/2017-08-08/modularity_wg.2017-08-08-14.00.log.html Meeting summary --- * Roll Call (nils, 14:00:29) * Agenda (nils, 14:03:56) * Module naming policies (nils, 14:03:56) * Host & Platform availability (nils, 14:03:56) * Fedora 27 modular design (nils, 14:03:56) * Modular Atomic Host (nils, 14:03:56) * Modular Guidelines and Review Process (nils, 14:04:37) * Module naming policies (nils, 14:05:05) * LINK: https://pagure.io/modularity/pull-request/43 Module naming policy proposal (contyk, 14:06:40) * Host & Platform availability (nils, 14:08:14) * The very first builds of Host & Platform are now available and the initial test composes should be ready later today. (contyk, 14:14:56) * Images will be available once we have the new installer module available. The current Platform is messy and will be sanitized over the next week or two. (contyk, 14:15:29) * Fedora 27 modular design (nils, 14:15:42) * We will be going through the list of required modules for Fedora 27, their built-time and runtime dependencies and deciding what buckets they should go into. (contyk, 14:25:44) * We will also generate module templates for upstream maintainers willing to help us with implementation, to guide them (contyk, 14:26:11) * LINK: https://github.com/fedora-modularity/dependency-report A helper module dependency reporting tool. WIP. (contyk, 14:26:27) * ACTION: langdon to file a ticket in pungi(?) backlog to create a modulemd -> pungi config generator (langdon, 14:44:57) * Modular Atomic Host (nils, 14:46:03) * LINK: https://pagure.io/atomic-wg/issue/312 Experiment with building Atomic Host out of a module (contyk, 14:46:36) * Expect a modular Atomic Host prototype within the next two weeks! (contyk, 14:50:51) * Modular Guidelines and Review Process (nils, 14:52:50) * fedora council approved fesco approving the initial guidelines and process, after that this group will be responsible for maintaining them (langdon, 15:00:05) * contyk, nils and geppetto are updating the spec (based on a couple missing items from the last week or so) and associated guidelines to be as close as possible (langdon, 15:00:23) * ACTION: langdon shooting to file a ticket with fesco by the end of the week to review it at the next fesco meeting next friday (langdon, 15:00:40) * ACTION: needs to get the bz product created to file review requests.. which I don't think we have done or assigned anyone too (langdon, 15:00:50) * ACTION: needs to identify any other gaps in the process that I am not thinking of (langdon, 15:01:03) Meeting ended at 15:04:11 UTC. Action Items * langdon to file a ticket in pungi(?) backlog to create a modulemd -> pungi config generator * langdon shooting to file a ticket with fesco by the end of the week to review it at the next fesco meeting next friday * needs to get the bz product created to file review requests.. which I don't think we have done or assigned anyone too * needs to identify any other gaps in the process that I am not thinking of Action Items, by person --- * langdon * langdon to file a ticket in pungi(?) backlog to create a modulemd -> pungi config generator * langdon shooting to file a ticket with fesco by the end of the week to review it at the next fesco meeting next friday * **UNASSIGNED** * needs to get the bz product created to file review requests.. which I don't think we have done or assigned anyone too * needs to identify any other gaps in the process that I am not thinking of People Present (lines said) --- * contyk (100) * langdon (57) * nils (57) * asamalik (29) * jkaluza (29) * threebean (26) * zodbot (15) * ttomecek (14) * tflink (3) * geppetto (2) * sgallagh (1) * jkurik (1) * dgilmore (1) * mikedep333tflink (0) Generated by `MeetBot`_ 0.1.4 .. _`MeetBot`: http://wiki.debian.org/MeetBot -- Nils Philippsen"Those who would give up Essential Liberty to Software Engineer purchase a little Temporary Safety, deserve neither Red Hat Liberty nor Safety." -- Benjamin Franklin, 1759 PGP fingerprint:C4A8 9474 5C4C ADE3 2B8F 656D 47D8 9B65 6951 3011 ___ devel mailing list -- devel@lists.fedoraproject.org To unsu
Re: [Modularity]: Service levels and EOL expectations?
On 08/07/2017 03:58 PM, Matthew Miller wrote: On Mon, Aug 07, 2017 at 02:10:23PM -0400, Stephen John Smoogen wrote: I still don't see how this is going to work with a tree of Service Levels and Lifetimes. Any module can not give a SL greater than the lowest SL and the shortest lifetime that any package in it is going to agree to. [EG if I am packaging up a wordpress module and glibc is on a 18 month lifetime but openssl is meh upstream always.. then unless I am going to maintain openssl myself with its own fork... my module is going to be 'meh upstream always'. If my module pulls in enough stuff to make it work, I am going to be dealing with the need of a lawyer to figure out which SL's and lifetimes are binding and what ones are not. Yeah, that would get crazy fast. The 6 month granularity proposal should help*some*, and we should probably go into this carefully. Technically, the SL for the module could have the narrow meaning referring to the module only, and the tools could follow the chain of dependencies and display it as: Service Level: 6 months (effective SL: 3 months/EOL: 2017-11-08 due to dependency on openSSL) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [Modularity]: Service levels and EOL expectations?
> Is there an active plan on figuring out these Service Levels? Is there > a ticket? Is there a specific person who owns this? I think we need at > least a preliminary understanding of what goes here for the F27 beta > (freeze on Sept. 9th, so... I guess by then?) Thanks for starting this. I'm not aware of a ticket or a responsible party at this point. +1 to working towards formalizing this. > I'm assuming "Service Levels" will include options like: > > * This module strives to be very stable and we will backport patches > > * This module tries to be stable but occasionally we may make breaking > changes with FESCo approval when it's the only option for a security > update (this matches the current Fedora updates policy at > https://fedoraproject.org/wiki/Updates_Policy#Security_fixes) > > * This module promises only a subset of functionality to remain the > same. For example, it will include a certain command line program but > doesn't promise that output of that program will aways be identical. > > * This is a development-stream module which makes no promises! (E.g, it is > Rawhide.) > > Is that the kind of thing others are expecting? Or was this to be more > like "security", "security+bugfix", > "security+bugfix+enhancements"? FYI - we had to put some initial SL values into the database to seed the branch migration. Our existing 'master', fedora, and epel branches needed values in the new database. Here are the ones we added: - rawhide - Our thought was that the 'rawhide' SL would signal whatever rawhide means today: more or less tracking upstream. The master branch of all packages got this SL applied in our migration. - bug_fixes - See security fixes below. - security_fixes - This, along with bug_fixes, was a generic SL that got applied to all "fedora" branches (f26, f25, etc..). - stable_api - This one got applied to all of the epel branches. https://pdc.stg.fedoraproject.org/rest_api/v1/component-sla-types/ The real meaning of these values isn't documented anywhere and so they should be taken with a grain of salt. They are something that we can and should change if FESCo (or releng?) or the packager group at large decide to organize our SLs along some other basis. > *Or*, is it something like "best effort", "sig maintained", > "core > critical path", "edition critical path", "spin critical path"? > > I can see the first idea (the * points above) as most useful to > end-users. The third idea is more useful for *us*. > > I'd also like to propose a policy for EOLs. I assume that some modules > will have undefined EOLs, and I think that's okay. There should be some > mechanism and rules for adding one (amount of notice expected, what to > do if we can't meet that expectation, how the tools will present the > addition of an EOL). When a specific EOL is given, though, I'd like a > rule which says that that EOL is aways a month after the planned > traditional Fedora release dates — so, June and November, basically. > (We could choose something like "The last Tuesday in June or November"; > I don't care specifically.) Also, EOL should be treated as a "no sooner > than" date, so if we slip the corresponding release, we could add a > week or two to the upcoming module EOLs. > > That way, users and admins aren't treated to an explosion of arbitrary > days where action is needed to stay on a current stream. Instead, they > can plan for annual upgrades as we do now. (I also expect the > "platform" module to follow the current Fedora release cycle, right?) > > Perhaps there could be an exception made to this rule for modules with > the "development stream" Service Level. Or, maybe those would just have > no defined EOL — like Rawhide today. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Arbitrary Branching and solving the "Change Checkpoint? Better hit the gas!" problem
On Mon, Aug 07, 2017 at 07:33:21AM -0400, Josh Boyer wrote: > On Sat, Aug 5, 2017 at 10:57 AM, Matthew Miller > wrote: > > Our Change process has the basic assumption that if a Change isn't > > working, we will be able to back out. But, in practice, when there are > > problems, we often find it that it's easiest to shrug and go forward, > > scrambling to fix problems in the change itself as well as any fallout. > > I won't say this is a _failure_ exactly — with the Change process, at > > least, we do have that checkpoint where we know the scramble is needed > > rather than waiting until the very last minute. But, especially if we > > are serious about a six-month schedule, it'd be _better_ if we could > > simply decide at the Change Checkpoints whether to include the feature > > at all. > > > > Arbitrary branching and Modularity give us a way to do this. We can, at > > any time, say "this release includes this set of modules" (see > > https://docs.pagure.org/modularity/design/constructing/compose-distribution.html > > and > > https://docs.pagure.org/modularity/design/constructing/back-together.html). > > > > I'm really liking what I'm seeing from the Boltron demo, questions > > about how to actually manage modules as an end user notwithstanding. If > > we can deliver this with minimal end-user disruption and yet give > > ourselves capabilities like this, it's a huge success. > > > > (Aside: This possibly involves what the Boltron walkthrough calls > > "system profiles". Modularity team! This isn't in the docs yet. Can you > > clarify?) > > Hm. I agree entirely with you, but when reading this I thought of > another possibility. I think modularity gives people the option for a > rolling release, where we don't have to make release == "collection of > specific module versions". If that's one of the outcomes, then > Changes simply become "this module version is available". I agree. Once everything is modular, distro-wide release changes stop making sense. Instead we should be proposing changes for modules -- announcements of streams and such. Fedora release could/should still be driven by a set of low level modules, such as Platform and a few more tightly linked with it. Would it make sense to split the compose into two or more in this case? "Platform and stuff" + "Other, mostly independent stuff"? P signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Long time for package to be tagged into Rawhide
On Tue, Aug 08, 2017 at 01:33:31PM +0100, Peter Robinson wrote: > On Tue, Aug 8, 2017 at 12:31 PM, Richard W.M. Jones wrote: > > > > I built this one 3½ hours ago: > > > > https://koji.fedoraproject.org/koji/buildinfo?buildID=953201 > > > > A whole series of builds depend on this but I'm still waiting for it > > to get into the buildroot ... Can something be done to speed this up? > > No, the mass rebuild is being signed into rawhide, sorry but 1000s of > packages takes time, current queue is ~2450, it's coming down but > signing isn't the fastest. Fair enough. I realized after that I could build it again just into the f27-ocaml2 side tag and that works ... Thanks, Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Long time for package to be tagged into Rawhide
On Tue, Aug 8, 2017 at 12:31 PM, Richard W.M. Jones wrote: > > I built this one 3½ hours ago: > > https://koji.fedoraproject.org/koji/buildinfo?buildID=953201 > > A whole series of builds depend on this but I'm still waiting for it > to get into the buildroot ... Can something be done to speed this up? No, the mass rebuild is being signed into rawhide, sorry but 1000s of packages takes time, current queue is ~2450, it's coming down but signing isn't the fastest. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Long time for package to be tagged into Rawhide
I built this one 3½ hours ago: https://koji.fedoraproject.org/koji/buildinfo?buildID=953201 A whole series of builds depend on this but I'm still waiting for it to get into the buildroot ... Can something be done to speed this up? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Pagure over dist-git: what changes?
Hi, How can one request new package for multiple repoes at once like it was possible with pkgdb, is this possible with this new tool? ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
