future development/maintenance of rpkg-util
Hello, I need to step down as a developer/maintainer of rpkg-util (https://pagure.io/rpkg-util) as I no longer have the incentive or knowledge required to maintain the project well enough. The purpose of this tool was to provide a distro-agnostic rpm package manager - that translates to: 1) it is able to communicate with any dist-git instance out there 2) it requires zero initial setup, i.e. one just needs to place a valid spec file into a git repo for rpkg command to be able to work with that repo (and with that package consequently) 3) the git repo with the spec file doesn't need to have the standard dist-git layout - context of the spec-file can be whatever as long as the spec properly references the sources 4) it additionally provides set of preproc (very simple templating language) macros that enable auto-generation of source tarball(s) and rpm metadata (Name, Version, Release, VCS, changelog, ...) "on-demand" from the content of the repository (i.e. source files) and its metadata (i.e. git history). "On-demand" means that these are only computed and substituted into the spec file when needed (e.g. srpm or rpm is built). There are two other ideas that drove the development except "the distro-agnostic package manager" thing: * provide the simplest possible way for upstreams to make rpm packages out of their application sources * do that while keeping all the standard workflows of a Fedora packager working (i.e. upload sources, download sources, build srpm, send package for a build into a build system) So rpkg-util fuses these two seemingly different things into a single one. I also had some other ideas, i.e. how to implement creating distro-specific variation on this tool that doesn't use inheritance but instead configuration and pluggable commands but this hasn't been implemented yet. There is also some potential for extending the macro library that rpkg-util offers or even the preproc (the bash-based templating language) syntax to provide a more full-fledged templating language (preproc is supposed to be non-specific to rpm btw. even though it is currently part of the rpkg-util git repo). It has found its use mainly in Copr in SCM built method. I don't know about uses outside of that scope. The project is in a very good shape currently with only these reported issues: https://pagure.io/rpkg-util/issues, which might change with time if it is unmaintained. So if anyone from RH or outside would like to step-in and take maintenance / further development of this project, that would be welcome. I would probably stick around to some extent and provide any guidance if needed. Best regards clime ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: rpkg-3.rc2 built into rawhide
A bit of heads up for anyone who uses custom/user macros with
rpkg(-util):
https://docs.pagure.org/rpkg-util/v3/macro_reference.html#user-defined-macros
With version 2, it was enough to have `rpkg.macros` file in the
directory where rpkg is executed and the custom macros were
automatically loaded from there.
In version 3, the path to file with the custom macros needs to be
specified through rpkg.conf file placed at the git top-level
directory, e.g.
$ cat some-git-repo/rpkg.conf
[rpkg]
user_macros = "${git_props:root}/rpkg.macros"
iff the rpkg.macros file is also placed at the top-level git directory
(i.e. next to rpkg.conf).
This will become relevant (for SCM rpkg srpm build method) with f35
builders in Copr.
---
Also if anyone is using the rpkg command-line tool locally, replacing
/etc/rpkg.conf with /etc/rpkg.conf.rpmnew will be needed.
Best regards
clime
On Thu, 13 May 2021 at 15:28, clime wrote:
>
> Hello,
>
> I have built rpkg-3.rc2 (package sources:
> https://src.fedoraproject.org/rpms/rpkg-util) into rawhide yesterday.
> It should be now available for installation.
>
> It is a complete rewrite of version 2. It aims to be a tool with
> minimal deps that allows people to easily transform git repositories
> containing spec files into rpm packages.
>
> Historically, the goal of this tool was to provide a single client to
> all dist-git instances (i.e Fedora, CentOS, maybe something else too).
> This was the idea when I started to work on it still in RedHat under
> the lead of Miroslav Suchý. We wanted to tweak the rpkg script
> provided by python-rpkg lib (https://pagure.io/rpkg) for that purpose.
> However, that script was soon being discontinued. There I created a
> separate project (rpkg-util) which would be providing the script so
> that I can continue in my goal.
>
> Eventually, I realized that some steps rather need to be taken
> server-side (https://github.com/release-engineering/dist-git) for a
> generic client to exist so that I did and then it was about adoption
> of the dist-git package (which provides the unified interface) by
> various dist-git instances.
>
> Meanwhile, I was thinking what to do next with rpkg while being
> heavily influenced by Copr that I primarily worked on at that time and
> also by discussion on devel list about automatically generating
> changelog and release from git. I realized there is an opportunity to
> provide a solution for this in the generic dist-git client (rpkg
> executable) and that it was not a bad place for it.
>
> This problem was then and even today solved by basically sedding spec
> file with regexps which I thought is kind of hacky and that it could
> be useful to have a library for common substitutions anyway so that
> people can reuse it. So a simple spec templating language was born
> (https://docs.pagure.org/rpkg-util/v3/spec_templates_from_scratch.html)
> and also a simple library of macros
> (https://docs.pagure.org/rpkg-util/v3/macro_reference.html) that use git
> content and metadata to render spec file content or parts of it. It
> now allows not only to generate changelog or release but also e.g. to
> generate a source tarball from a git submodule placed next to the spec
> file.
>
> There is also the possibility for users to define their own macros and
> therefore customize git->rpm transform in any way.
>
> You can try this now in rawhide (`dnf install rpkg`) or also in Copr
> (SCM rpkg build method) but note that in Copr, there is still v2 so
> there might be some differences. You might also need to install
> python3-setuptools - that package seems to be required by
> python3-munch but is not being brought in automatically at the moment.
>
> You can use rpkg tool similarly to fedpkg to work with Fedora dist-git
> packages (except that `rpkg build` command builds in Copr and the
> command set is mostly trimmed-down) or you can use it to work with
> those git layouts that contain unpackaged sources together with spec
> file (or rather spec file template) or link to the upstream sources
> through a git submodule.
>
> Best regards
> clime
>
> P.S.: the project is hosted here: https://pagure.io/rpkg-util
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
rpkg-3.rc2 built into rawhide
Hello, I have built rpkg-3.rc2 (package sources: https://src.fedoraproject.org/rpms/rpkg-util) into rawhide yesterday. It should be now available for installation. It is a complete rewrite of version 2. It aims to be a tool with minimal deps that allows people to easily transform git repositories containing spec files into rpm packages. Historically, the goal of this tool was to provide a single client to all dist-git instances (i.e Fedora, CentOS, maybe something else too). This was the idea when I started to work on it still in RedHat under the lead of Miroslav Suchý. We wanted to tweak the rpkg script provided by python-rpkg lib (https://pagure.io/rpkg) for that purpose. However, that script was soon being discontinued. There I created a separate project (rpkg-util) which would be providing the script so that I can continue in my goal. Eventually, I realized that some steps rather need to be taken server-side (https://github.com/release-engineering/dist-git) for a generic client to exist so that I did and then it was about adoption of the dist-git package (which provides the unified interface) by various dist-git instances. Meanwhile, I was thinking what to do next with rpkg while being heavily influenced by Copr that I primarily worked on at that time and also by discussion on devel list about automatically generating changelog and release from git. I realized there is an opportunity to provide a solution for this in the generic dist-git client (rpkg executable) and that it was not a bad place for it. This problem was then and even today solved by basically sedding spec file with regexps which I thought is kind of hacky and that it could be useful to have a library for common substitutions anyway so that people can reuse it. So a simple spec templating language was born (https://docs.pagure.org/rpkg-util/v3/spec_templates_from_scratch.html) and also a simple library of macros (https://docs.pagure.org/rpkg-util/v3/macro_reference.html) that use git content and metadata to render spec file content or parts of it. It now allows not only to generate changelog or release but also e.g. to generate a source tarball from a git submodule placed next to the spec file. There is also the possibility for users to define their own macros and therefore customize git->rpm transform in any way. You can try this now in rawhide (`dnf install rpkg`) or also in Copr (SCM rpkg build method) but note that in Copr, there is still v2 so there might be some differences. You might also need to install python3-setuptools - that package seems to be required by python3-munch but is not being brought in automatically at the moment. You can use rpkg tool similarly to fedpkg to work with Fedora dist-git packages (except that `rpkg build` command builds in Copr and the command set is mostly trimmed-down) or you can use it to work with those git layouts that contain unpackaged sources together with spec file (or rather spec file template) or link to the upstream sources through a git submodule. Best regards clime P.S.: the project is hosted here: https://pagure.io/rpkg-util ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: libravatar ported to Fedora's AWS
Thank you all for the kind words! clime On Thu, 18 Mar 2021 at 14:49, David Duncan wrote: > > Great News! Much appreciated! > > David Duncan > http://about.me/davdunc > > > On Wed, Mar 17, 2021 at 9:37 PM Christoph Karl wrote: > > > > Am 14.03.21 um 23:32 schrieb clime: > > > Hello, > > > > > > I have just finished port of libravatar.org service to server provided > > > by Fedora. Big thanks to the Fedora project for sponsoring libravatar. > > > Avatars in pagure.io, src.fp.o, Bodhi should now load much faster. > > > > +1 > > > > Christoph > > ___ > > devel mailing list -- devel@lists.fedoraproject.org > > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > > Fedora Code of Conduct: > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > > List Archives: > > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > > Do not reply to spam on the list, report it: > > https://pagure.io/fedora-infrastructure > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
libravatar ported to Fedora's AWS
Hello, I have just finished port of libravatar.org service to server provided by Fedora. Big thanks to the Fedora project for sponsoring libravatar. Avatars in pagure.io, src.fp.o, Bodhi should now load much faster. Best regards! clime ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora 35 Change: rpmautospec - removing release and changelog fields from spec files (System-Wide Change proposal)
On Sun, 14 Feb 2021 at 22:16, Kevin Fenzi wrote: > On Sun, Feb 14, 2021 at 09:39:01PM +0100, Miro Hrončok wrote: > > > > That still means a particular change needs to "bump" the release number > to a > > specific value and hence it generates conflicts/mismatch when: > > > > - two or more PRs are opened at the same time > > - individual commits are cherry-picked between branches > > > > Hence, not sure if it actually solves some problem. > > See, I knew there would be problems. ;) > > Anyhow, I think it's a noble goal if we can have src.rpms still provide > all the information needed to contibute (and I know, the shipped rpms > from this will match the build, but not whats in git). Conflicts are not the problem as the release would be derived from the dir structure and in this structure the content is separate. There is a more subtle problem that you should include branchname in the structure because release 2 in one branch can mean something else than release 2 in another (this could lead to git conflicts but that's rather coincidental consequence). Just wanted to say that your proposal would solve that particular problem as well. > > > kevin > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Ars claims: Fedora 32 is sluggish
On Thu, 4 Feb 2021 at 21:12, clime wrote: > On Thu, 4 Feb 2021 at 19:36, Matthew Miller > wrote: > >> On Thu, Feb 04, 2021 at 10:33:37AM -0700, Nathanael D. Noblet wrote: >> > > I genuinely wonder if this is due to the launch animation. I know that >> > > subjectively for myself using the Impatience to triple the speed makes >> > > my desktop feel more snappy. >> > >> > Sorry, what is 'the Impatience'? How does it improve the desktop >> > performance? >> >> It is this extension: >> https://github.com/timbertson/gnome-shell-impatience >> >> It does not _actually_ improve performance. It gives you a slider which >> can >> be used to double or triple (or whatever) the speed of the animations, >> which >> has — to my easily-tricked brain — the effect of making the desktop _seem_ >> more responsive. Try it and see if it works for you too! >> > > Thanks, it helps even for going to overlay and back and for switching > workspaces. > I now have more time to procrastinate, which is great! > > >> >> -- >> Matthew Miller >> >> Fedora Project Leader >> ___ >> devel mailing list -- devel@lists.fedoraproject.org >> To unsubscribe send an email to devel-le...@lists.fedoraproject.org >> Fedora Code of Conduct: >> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ >> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines >> List Archives: >> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org >> > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Ars claims: Fedora 32 is sluggish
On Thu, 4 Feb 2021 at 19:36, Matthew Miller wrote: > On Thu, Feb 04, 2021 at 10:33:37AM -0700, Nathanael D. Noblet wrote: > > > I genuinely wonder if this is due to the launch animation. I know that > > > subjectively for myself using the Impatience to triple the speed makes > > > my desktop feel more snappy. > > > > Sorry, what is 'the Impatience'? How does it improve the desktop > > performance? > > It is this extension: https://github.com/timbertson/gnome-shell-impatience > > It does not _actually_ improve performance. It gives you a slider which can > be used to double or triple (or whatever) the speed of the animations, > which > has — to my easily-tricked brain — the effect of making the desktop _seem_ > more responsive. Try it and see if it works for you too! > Thanks, it helps even for going to overlay and back and for switching workspaces. > > -- > Matthew Miller > > Fedora Project Leader > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: [Help wanted] Setting vi/view/vim via alternatives
On Sat, 30 Jan 2021 at 20:03, Vitaly Zaitsev via devel wrote: > > On 30.01.2021 18:50, clime wrote: > > Hey, what do you mean by this? > > https://docs.fedoraproject.org/en-US/fedora-coreos/alternatives/ > > > Why should scriplets usage be incompatible with immutable Fedora > > releases? > > rpm-ostree doesn't execute any scriptlets. It doesn't seem true per what Fabio linked: https://github.com/coreos/fedora-coreos-tracker/issues/703 (and the fix https://src.fedoraproject.org/rpms/wireshark/pull-request/5#request_diff) That rather suggests they need the scriplets to be OSTree-compatible. I think the main reason for the problem with alternatives is really just that it tries to put its db under /var/, which is not supported as /var is not synced by OSTree. By the way, this is really cool thing: https://docs.fedoraproject.org/en-US/packaging-guidelines/EnvironmentModules/ Not usable for this particular case but really cool. It's interesting that it itself uses alternatives for switching between the lmod and the env implementation. > > -- > Sincerely, >Vitaly Zaitsev (vit...@easycoding.org) > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: [Help wanted] Setting vi/view/vim via alternatives
On Sat, 30 Jan 2021 at 19:48, Fabio Valentini wrote: > > On Sat, Jan 30, 2021 at 7:24 PM Peter Boy wrote: > > > > > > > > > Am 30.01.2021 um 17:45 schrieb Vitaly Zaitsev via devel > > > : > > > > > > On 30.01.2021 16:58, Peter Boy wrote: > > >> But it’s perfectly usable for Fedora Workstation or Server and almost > > >> indispensable for some development projects, e.g. Java (and vi/vim for a > > >> terminal environment). Why should alternatives not be usable there? Or > > >> what is a suitable and adequate replacement? > > > > > > https://docs.fedoraproject.org/en-US/packaging-guidelines/Alternatives/ > > > > Thanks for the info. Unfortunately, I don’t see a connection to immutable > > Fedora (it is about drop-in, user configurable, etc). Or do I miss > > something? > > If you read the Packaging Guidelines, they actually explicitly mention > that vi / vim are a bad example for using the alternatives system - > because they're not drop-in replacements. > > Additionally, as far as I know, OSTree based Fedora variants do not > execute any RPM scriptlets, but implement their own handling of e.g. > ldconfig and such things. > And alternatives is definitely not compatible with OSTree - according > to these bug reports, at least Java alternatives are broken - > apparently primarily because OSTree stores configuration in /var > instead of /etc: I think you meant: "...because alternatives stores configuration in /var instead of /etc" > > - https://bugzilla.redhat.com/show_bug.cgi?id=1657367 > - https://github.com/coreos/rpm-ostree/issues/1614 > > > Fabio > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Policy proposal (draft): Don't push knowingly broken or work-in-progress work to dist git
On Sat, 30 Jan 2021 at 01:28, Kevin Kofler via devel wrote: > > Michael Catanzaro wrote: > > Alternative: use automated reverts instead of force pushes, and don't > > worry about maintaining a clean history. > > Sure, it is possible to make an implementation with lower quality of > implementation with possibly less work, by omitting the force pushes and the > smart "fedpkg build" behavior. > > That said, I think you will find that reverts are actually more work to get > right in complex cases such as multi-commit pushes, possibly even with merge > commits, than a simple: > git reset --hard $last_successful_build > git push -f > which only needs the CI to be exempted from the git hook banning force > pushes. So if some other maintainer pushes his work to the server meanwhile, this will just delete his work? Or what's the idea here? > > Kevin Kofler > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: [Help wanted] Setting vi/view/vim via alternatives
On Sat, 30 Jan 2021 at 16:08, Vitaly Zaitsev via devel wrote: > > On 29.01.2021 09:49, Zdenek Dohnal wrote: > > I'm currently trying to rewrite the current shell aliases for making > > Vi/View/Vim use the correct compiled binary based on which Vim package > > is installed. > > Alternatives are not suitable for Fedora, because they will break > immutable Fedora releases due to scriptlets usage. Hey, what do you mean by this? Why should scriplets usage be incompatible with immutable Fedora releases? I guess thousands of rpms use some kind of scriplets in spec? Otherwise, alternatives subcommands alter stuff under `/etc/alternatives` and `/var/lib/alternatives`. Both places should be writable on immutable releases. > > -- > Sincerely, >Vitaly Zaitsev (vit...@easycoding.org) > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: How did this file end up in the lookaside cache?
On Tue, 12 Jan 2021 at 01:14, Cristian Morales Vega wrote: > > Not sure exactly when, but it looks like me creating > https://src.fedoraproject.org/rpms/doxygen/pull-request/9 has > triggered a process uploading the tarball to the lookaside cache. > > The tarball inside > https://kojipkgs.fedoraproject.org//work/tasks/5674/59445674/doxygen-1.9.1-1.fc34.src.rpm > is fine. But when I later tried to build the same sources in copr I > got a 404 error when downloading the tarball from the lookaside cache > (https://download.copr.fedorainfracloud.org/results/reddwarf/personal/srpm-builds/01876339/builder-live.log.gz). > The thing is that > https://src.fedoraproject.org/lookaside/pkgs/doxygen/doxygen-1.9.1.src.tar.gz/sha512/a84fbea1874921317b58345c53fc4eac0382c9e593f0e1ee899a31e67ead3fd12b2da8145b37e2e09d665e28d060e6717c92de7e8d0a31fc5f24fdcc4715f54d/doxygen-1.9.1.src.tar.gz > is a 19 MiB monster (instead of the 5 MiB of the real tarball) with, > obviously, the wrong hash. > > How did this happen? Is it "expected" even if I don't understand it? > Is it a bug somewhere? It seems, the package maintainer himself updated to 1.9.1 and uploaded the new sources in about the same time (maybe slightly later) you opened the pull request. https://src.fedoraproject.org/rpms/doxygen/commits/master That probably led to the confusion. Lookaside cache is not populated just by opening a pull request :). Best Regards clime > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
...snip... Btw. I posted a long comment here: https://pagure.io/fesco/issue/2532 basically trying to explain the proposal more and mention the use-cases where it would be useful. So, please, read if you are interested. I guess, if there is a further discussion it should be probably carried out here so that it is available for everyone. One more thing... it's interesting to note that 12 years ago, we had a very similar discussion to what we have today: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/I35MFMMIDO5FFY3ZFCXGE26PREPPWPCA/#DEJUEWXMAZND626D7ZOSOIFHGWSYGPAG - the problem of generating release by using build system vs. using just cvs/git was discussed there (Colin Walters vs. Jesse Keating). It would be interesting to know what people discussing the topic in the thread think about it today. clime ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
On Mon, 4 Jan 2021 at 20:43, Dan Čermák wrote: > > clime writes: > > > ...snip... > >> > > >> > $ preproc-rpmspec pkg.spec.rpkg # prints rendered spec to stdout, > >> > pkg.spec.rpkg is a spec template > >> > >> This would be a viable workaround, but a workaround nevertheless. Since > >> I am not frequently rebuilding Fedora rpms outside of mock, koji & copr, > >> I cannot tell how much of a show-stopper that is though. At least I > >> think that the benefits of the change need to be pretty big to outweigh > >> this potential downside. > > > > If I may ask, why do you think using preproc-rpmspec is a workaround? > > I consider it a workaround, because it adds another step to the > packaging process outside of koji & copr: it still works, but you have > to go through extra steps. Yes, there is an extra step that would be wrapped in higher-level tooling. But when you said "workaround", I was thinking that you actually saw the correct solution because "workaround" is imho used usually when someone can't or don't want to solve things the right way so he/she takes a shortcut. So I was curious what you think is "the right way" here. > > > Cheers, > > Dan ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: How to easily automate test builds in a COPR project
On Thu, 31 Dec 2020 at 14:59, Richard Shaw wrote:
>
> I maintain a suite of ham radio related packages. The developer is very
> active and often creates test versions adding and incrementing the "tweak"
> part of the version which is removed for the full releases and the patch
> level incremented.
>
> Currently I'm just trying to keep up with them by hand using pagure forks of
> the official repos so I don't accidentally pollute SCM with the changes and
> build them in COPR.
>
> Things I need to manage automagically:
> 1. Monitor the test URLs to look for new versions.
>
> I could write a bash script for this and add a cron or systemd timer but I
> was hoping for something that took less time as I don't have a lot of that :)
>
> Would it be permissible to create a -testing entry in
> release-monitoring.org?
>
>
> 2. Trigger a "fedpkg clone" and add a tweak version.
>
> This could probably be managed with macros easy enough, %{?tweak}, or
> something like that. And then use a script to substitute into "%global tweak
> ..."
>
>
> 3. I need to download the files from a different location.
>
> %if %{?tweak}
> ... use difference Source0?
>
>
> 4. Build the packages in COPR.
>
> Easy enough using a bash script but is there a better way?
There would be a very neat way if "Enable Spec File preprocessing"
change was accepted but it looks like it won't happen. There are still
some options but they require keeping a spec file (template) outside
of Fedora DistGit (which should normally be the source for the spec
file). Actually, also rpkg-3 would be needed in Copr (currently rpkg-2
is there). So with rpkg-3, you could:
- have the spec file template e.g. in Pagure
- configure rpkg SCM method by using https URL of the spec file in
pagure and clone url of upstream (this is a thing currently
unsupported by rpkg-2)
- either ask the upstream developer to configure a webhook for you to
trigger a build in COPR upon a new commit or tag or configure
release-monitoring for that
With the aforementioned Fedora change, you could have just a single
spec file in Fedora DistGit and use it for official Fedora builds as
well as for upstream builds. But without that change, it is not
possible so you would need to take rendered spec files from the
template and load them into DistGit with an upstream version that you
would like to release into Fedora. That breaks Fedora DisGit
canonicity but it's the best way and many people do it anyway.
You can actually implement this workflow even today without rpkg-3 in
COPR by using COPR's custom srpm build method. There you could specify
a snippet which installs rpkg-3 into the chroot and then invokes
`rpkg` command on the spec in the pagure (again specified as https
URL) while operating on the upstream cloned repo. Of course, you could
also avoid using rpkg-3 and do something more manual on your own
(e.g. invoking git archive and sedding your spec file with the
produced sourcename).
Well, I probably didn't help you much
Anyway, best regards
clime
>
> Thanks,
> Richard
>
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
...snip... > > > > $ preproc-rpmspec pkg.spec.rpkg # prints rendered spec to stdout, > > pkg.spec.rpkg is a spec template > > This would be a viable workaround, but a workaround nevertheless. Since > I am not frequently rebuilding Fedora rpms outside of mock, koji & copr, > I cannot tell how much of a show-stopper that is though. At least I > think that the benefits of the change need to be pretty big to outweigh > this potential downside. If I may ask, why do you think using preproc-rpmspec is a workaround? > > > Cheers, > > Dan ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Popularity contest for Fedora
On Sun, 27 Dec 2020 at 17:41, Gary Buhrmaster wrote: > > On Sun, Dec 27, 2020 at 3:12 PM Matthew Miller > wrote: > > > It's been talked about before but no one has done it. > > There was also smolt, which collected some > system information (and could be extended > to collect more) However, not only did the > upstream die, follow-on proposals never > took off, and also opened the entire > can-of-worms regarding an opt-in data > collection mechanism (and it was agreed > by most it had to be opt-in) not being able to > provide useful data to actually make good > decisions on. It is also true that many wish > we did have sufficiently good data in order > to make good decisions. Rock, meet hard > place. I think we can simply parse server-side access logs to count package downloads, no? It won't be probably very precise but could be enough to give us a basic idea... clime > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: libmemcached replacement
On Tue, 22 Dec 2020 at 10:55, Remi Collet wrote: > > Hi, > > libmemcached exists in Fedora for years and is used by lot of projects > to handle communication with a memcached server. > > Sadly this project is dead: > https://launchpad.net/libmemcached/ > > Last version released in 2014 > and nearly no git activity since this > > > A fork now exists > https://github.com/m6w6/libmemcached/ > > Version 1.1.0beta1 is released and is design to > be a drop in replacement, with API and ABI compatibility > > > I've start working on a package update > and this will probably become the new upstream > for the fedora libmemcached package > > > Comment / idea ? For me, the biggest issue always is that I don't know if I should pick php-memcache or php-memcached plugin...is there any recommendation regarding this? Thank you! clime > > > Remi > > > P.S. of course, I'm mostly interested by the the PHP extension > which uses it https://pecl.php.net/package/memcached > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
On Fri, 18 Dec 2020 at 17:03, clime wrote: > > On Fri, 18 Dec 2020 at 16:23, James Szinger wrote: > > > > On Fri, 18 Dec 2020 00:51:49 +0100 > > clime wrote: > > > > > Well, the users here are still packagers here no? I thought the "User" > > > in the title means "end user" who shouldn't be affected by it. Maybe > > > Ben can clarify this. > > > > I am making a distinction between Fedora packagers who use the Fedora > > infrastructure to build RPMs for the Fedora repositories, and second > > and third party packagers who use their own infrastructure. External > > packagers count as ‘users’ for the purposes of change proposals, > > especially infrastructure changes such as this. Many change proposals > > have no effect for external packagers, or have the same effect as for > > the general user base. This proposal, however, seems potentially > > disruptive for external packagers, and I want to see those issues > > specifically addressed in the change proposal. > > > > I will advocate that external packagers are of strategic importance to > > the Fedora Project. The software they provide encourages the adoption > > of Fedora. They form a pool a potential Fedora packagers. They are > > the technical experts that support the local users. They are > > fundamental to the Fedora Mission: > > > > Fedora creates an innovative platform for hardware, clouds, and > > containers that enables software developers and community members > > to build tailored solutions for their users. > > > > Supporting external packagers drives the Four Foundations: Freedom, > > Friends, Features, and First. > > I agree the change proposal should address those issues. I have updated the change description (https://fedoraproject.org/wiki/Changes/Enable_Spec_File_Preprocessing) with the feedback I've been given in this thread. I have also expanded the "User experience" section so that it addresses the mentioned issues of having spec file templates (i.e. something which is not understood by rpmbuild) in dist-git instead of just plain spec files. Also some links to interesting past discussions about the dynamic changelog and release generation were added into the description (the problem actually has a very long history and was discussed many times). Hopefully, I didn't forget to mention something (will add it later if I did). Best regards! clime > > > > > Jim > > ___ > > devel mailing list -- devel@lists.fedoraproject.org > > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > > Fedora Code of Conduct: > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > > List Archives: > > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: our containers with alias vim=vi
I have just noticed in f33 container, this change was implemented: [root@9c06602a8aa6 ~]# vim /etc/yum.repos.d/x.repo No vim found, using vi, press ENTER to continue It's much much better, thank you a lot to the maintainer (probably Zdenek)! On Wed, 14 Oct 2020 at 13:29, Jonathan Wakely wrote: > > On 13/10/20 16:04 +0200, Zdenek Dohnal wrote: > >On 10/13/20 12:34 PM, Jonathan Wakely wrote: > >> On 13/10/20 07:45 +0200, Zdenek Dohnal wrote: > >>> > >>> On 10/12/20 5:15 PM, Joe Doss wrote: > >>>> On 10/12/20 1:50 AM, Zdenek Dohnal wrote: > >>>>> This would break using Vim when vim-minimal and vim-enhanced are > >>>>> installed (it would start Vi instead of typed Vim). To make it work, > >>>>> vim-minimal would have to conflict with vim-enhanced, which doesn't > >>>>> make > >>>>> sense - Vi and Vim binaries can exist together just fine. > >>>> > >>>> I have vim-enhanced and vim-minimal installed > >>>> > >>>> # rpm -qa |grep vim > >>>> vim-common-8.2.1770-1.fc33.x86_64 > >>>> vim-filesystem-8.2.1770-1.fc33.noarch > >>>> vim-minimal-8.2.1770-1.fc33.x86_64 > >>>> vim-enhanced-8.2.1770-1.fc33.x86_64 > >>>> > >>>> and when I type vi it launches vim. > >>> I'm sorry I forgot about this alias - yes, there is an alias which is > >>> applied when both - vim-minimal and vim-enhanced - are installed so it > >>> launches 'vim' when you type 'vi'. I would say less people will complain > >>> if something has more features than if it has less, so I'm content with > >>> this alias. > >>>> > >>>> # whereis vi > >>>> vi: /usr/bin/vi /usr/share/man/man1p/vi.1p.gz > >>>> /usr/share/man/man1/vi.1.gz > >>>> # /usr/bin/vi --version > >>>> VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Sep 29 2020 00:00:00) > >>>> > >>>> It doesn't look like these are existing together just fine. It seems > >>>> that vim takes over vi. Shouldn't these conflict and only one can be > >>>> installed over the other? > >>> 'Vi' as the original project is no longer (I'm not sure for how long) > >>> shipped in Fedora. 'Vi' we ship is just 'VIM' compiled with 'small' set > >>> of features (f.e. without syntax highlighting) to mimic the original > >>> 'Vi'. So if you run 'vi --version' it always shows 'VIM'. > >>>> > >>>>> In the end I find it incorrect to mislead users by default by telling > >>>>> them 'Vim works' but Vi is run instead - Vi and Vim don't have the > >>>>> same > >>>>> set of features, which may lead into bug reports caused by this > >>>>> mistake. > >>>> > >>>> Isn't that the reverse behavior detailed above? I type vi on Fedora > >>>> Workstation it launches vim? I assume this isn't causing bug reports. > >>> You're right, as I wrote above - aliasing vi->vim doesn't seem as a > >>> problem to me, but aliasing vim->vi as clime suggested can cause mislead > >>> for users. > >> > >> I would also appreciate if "vim" ran the executable installed by > >> vim-minimal. I frequently type "vim" on servers I don't own and then > >> grumble that it fails and I have to run "vi" instead. It **is** vim, > >> it's just not installed with that name. Insisting that users call it > >> vi when we know it's vim and they know it's vim seems unnecessary. > > > >It's Vim but with a different set of features - VIM compiled as Vi > >binary is trying to be small, kind of simulate Vi behavior without > >setting 'compatible'. > > > >Since you know it has less features, good for you. But unfortunately, > >not all users know - there was already a report about Vim missing > >highlighting, and the reporter was running /usr/bin/vi. So my aim is to > >prevent such a report. > > > >> > >> $ rpm -qf /usr/bin/vi > >> vim-minimal-8.2.1770-1.fc32.x86_64 > >> > >> Could vim-minimal and vim-enhanced both install the same > >> /etc/profile.d/vim.sh file that did something like this? > >Installing the same file would mean to set conflicts between those two > >package, wouldn't it? Or would you mind explaining how to achieve it? > > RPM allows a file to be owned by more than one package. Fedora has > allowed this for eight ye
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
On Sun, 20 Dec 2020 at 11:39, Miro Hrončok wrote: > > On 12/20/20 1:38 AM, clime wrote: > >> I view this proposal as a risk that the spec files will look a bit more > >> weird, and the spec files maintenance will start diverging too much. > >> Everything happening for an overestimated triviality as IMO > >> the release/changelog is [1]. > > Well, even if this change is accepted, it doesn't mean people will use > > the feature so if the feature is overkill or it is generally bad, it > > will just die on its own. > > No. In fact, even if one maintainer keep using this (e.g. you), as a > provenpackager I still need to be able to deal with that. So, while this is > "opt-in only" for the individual packagers, it impacts all our provenackagers > (and some of our downstreams as well). In some cases, the effect of this change on the work of ProvenPackagers will be positive. Manual release bumping (because of soname-bump or a hotfix change) is easier because you can just call `fedpkg tag` and write a message instead of manually messing with a spec file. And in case a script is used, then it's exactly the same amount of work (calling the script). > > As such, I think this is not worth it. > > -- > Miro Hrončok > -- > Phone: +420777974800 > IRC: mhroncok > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
On Sun, 20 Dec 2020 at 00:23, Pavel Raiskup wrote:
>
> On Thursday, December 17, 2020 8:05:40 PM CET Ben Cotton wrote:
> > https://fedoraproject.org/wiki/Changes/Enable_Spec_File_Preprocessing
> >
> > == Summary ==
> > This change should enable an opt-in spec file preprocessor in Fedora
> > infrastructure for the benefit of packagers. The preprocessor allows
> > some very neat tricks that were impossible before, for example
> > generate changelog and release automatically from git metadata or pack
> > the entire dist-git repository into an rpm-source tarball (effectively
> > allowing unpacked repos to live in DistGit).
>
> It would be nice to see this in some concrete example. E.g. in some
> 'private-rpkg-preview' branch in some of the existing Fedora packages, so
> we can make a clearer idea about what this causes with real spec file
> readability (diff) and the initial newcomer barrier. So we could e.g.
> checkout that branch, and try to build the package. And see the real
> pros/cons.
Is this example good?:
https://pagure.io/hello_rpkg/blob/master/f/hello_rpkg.spec.rpkg
It can be built in Copr by SCM rpkg method.
There is also:
https://pagure.io/hello_rpkg_release/blob/master/f/hello_rpkg.spec.rpkg
with a dynamic release, the macro used is actually v3 macro, not yet
supported by Copr.
v3 macros are being introduced by this change. One can try to build
that spec either by rpkg-util (from Copr repo referenced in the
change) or by mock with rpkg_preprocessor enabled
I think pros/cons can be also revealed by discussion which is perhaps
better because it is shared...
>
> > == Benefit to Fedora ==
> >
> > This change offers solution to some long-standing issues in Fedora
> > around packaging (i.e. automatic release and changelog generation)
>
> I personally wouldn't overestimate these issues, at least according to the
> questionnaire I tried to do some time ago [1, 2] not many maintainers were
> interested in the problem to even vote (and I was not surprised).
Well but there were also quite many mailing lists threads indicating
that people are interested in the topic. I cannot say if people are
interested or not...
>
> These problems have trivial work-arounds/solutions, discussed in [1].
Idk If I want to get into the whole theory behind this problem. If
there is a trivial solution to the problem (i.e. automatic release and
changelog in this case), then perhaps it should be stated here and it
can be taken into account when deciding about this change. I would
like to note that this change tries to solve the problem but also in a
way that allows for more applications in future (e.g. unpacked repos),
which might be considered a potential advantage.
>
> > while also offering some interesting future options (for example
> > unpacked dist-git repos).
>
> I think that it would be good to consider tito as alternative, when we
> speak about binding spec with git-archive feature. Do you think it would be
> possible to allow tito in future?
Maybe...If people want to use tito and ProvenPackagers and relengs are
okay with that...it's not really something I can decide.
> There's also some Packit-team feature named source-git, is this related?
It is related with regards to the "unpacked repos" that I described earlier.
> -
> Honestly, in general, I don't like tito, and I don't like rpkg much more.
> Both are probably better for upstream development and release processes
> (tito is more standard and convenient IMO) than some custom scripting.
> But people need to know deeply the use-cases (and even implementation) to
> compare.
>
> Also note that we used to have problems [IIRC 3, 4] in Copr builds from
> Fedora DistGit -- as the '{{{' syntax collided with some of the existing
> packages.
Yes, there were two of them (last time I checked):
python-dns-lexicon.spec - uses {{{ }}} in comments
python-suds.spec - uses {{{ }}} in changelog at one place
But the feature is opt-in so a maintainer can tweak his/her spec file if needed.
>
> I view this proposal as a risk that the spec files will look a bit more
> weird, and the spec files maintenance will start diverging too much.
> Everything happening for an overestimated triviality as IMO
> the release/changelog is [1].
Well, even if this change is accepted, it doesn't mean people will use
the feature so if the feature is overkill or it is generally bad, it
will just die on its own.
>
> [1]
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/2G6OSOSM76O2V6K4COIE2QHNXIBSXPFR/
> [2]
> https://docs.google.com/forms/d/183dSFIN-i9rauEZ0_gtDia7dzkeX-hzfX0ncpqFMYxw/edit#responses
> [3] https://pagure.io/copr/copr/issue/798
> [4] https://pagure.io/copr/copr/issue/1219
>
> Pavel
>
>
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
>
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
On Sat, 19 Dec 2020 at 20:57, clime wrote: > > On Sat, 19 Dec 2020 at 20:07, Dan Čermák > wrote: > > > > clime writes: > > > > > On Thu, 17 Dec 2020 at 22:04, James Szinger wrote: > > >> > > >> On Thu, 17 Dec 2020 14:05:40 -0500 > > >> Ben Cotton wrote: > > >> > > >> 1. How does this affect users who download, maybe modify, and rebuild > > >> the SRPM? Can they continue to use rpmbuid and mock as they have > > >> been? Does the SRPM contain the pre-processed or post-processed spec > > >> file? > > > > > > They can use mock if the preprocessing will be enabled for the > > > respective chroots where it is enabled in Koji/Fedora. > > > They can't directly use rpmbuild for those packages that contain the > > > macros. But they can use rpkg/fedpkg to do the work. > > > Or preprocess spec first and then use rpmbuild. I am aware this is a > > > negative point of this change. > > > > This is a pretty big downside imho, as that means that building Fedora > > packages that use these new kinds of macros in other build systems will > > become impossible or at the very least, very, very difficult. There is > > quite some development going on in OBS (afaik e.g. Igor exported all > > Fedora Rust rpms to OBS for automated rebuilds) and enabling this > > preprocessing will make these packages FTBFS in OBS. > > > > It depends on how the srpms are being built and if Fedora DistGit is > used directly as the source (as Adam has said also). If there is such > a possible breakage, we can look at fixing it in advance. > > The tooling that implements preprocessing has minimal requirements > (git or git-core, bash, python, libgit2-devel, rpm-devel) so there > should be a very low barrier for entry for any environment that would > need it. > > > Don't get me wrong, I am not opposed to this proposal per-se. But as far > > as I recall, many people were pretty upset about modular packages being > > effectively only buildable in Fedora's infra and nowhere else. And I'd > > very much like not to repeat this. > > > > > While having an option to use rpmbuild directly to build srpm/rpm from > > > a dist-git repo is nice, I would say that fedpkg or mock are the main > > > interfaces to do this. > > > I know this answer won't satisfy everyone. > > > > Indeed. I think there *should* be at least a way how to produce a srpm > > that can be rebuild *without* having access to Koji, mock and fedpkg > > (ideally by our own infrastructure). > Well, also once you produce an srpm, it will be just a generic srpm that can be rebuilt in any rpm-compatible system. > Well, that excludes lots of options already :). One can also use > preproc-rpmspec tool to get a rendered spec file (this is what mock > uses). > > $ preproc-rpmspec pkg.spec.rpkg # prints rendered spec to stdout, > pkg.spec.rpkg is a spec template > > > > > > > Cheers, > > > > Dan ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
On Sat, 19 Dec 2020 at 20:07, Dan Čermák wrote: > > clime writes: > > > On Thu, 17 Dec 2020 at 22:04, James Szinger wrote: > >> > >> On Thu, 17 Dec 2020 14:05:40 -0500 > >> Ben Cotton wrote: > >> > >> 1. How does this affect users who download, maybe modify, and rebuild > >> the SRPM? Can they continue to use rpmbuid and mock as they have > >> been? Does the SRPM contain the pre-processed or post-processed spec > >> file? > > > > They can use mock if the preprocessing will be enabled for the > > respective chroots where it is enabled in Koji/Fedora. > > They can't directly use rpmbuild for those packages that contain the > > macros. But they can use rpkg/fedpkg to do the work. > > Or preprocess spec first and then use rpmbuild. I am aware this is a > > negative point of this change. > > This is a pretty big downside imho, as that means that building Fedora > packages that use these new kinds of macros in other build systems will > become impossible or at the very least, very, very difficult. There is > quite some development going on in OBS (afaik e.g. Igor exported all > Fedora Rust rpms to OBS for automated rebuilds) and enabling this > preprocessing will make these packages FTBFS in OBS. > It depends on how the srpms are being built and if Fedora DistGit is used directly as the source (as Adam has said also). If there is such a possible breakage, we can look at fixing it in advance. The tooling that implements preprocessing has minimal requirements (git or git-core, bash, python, libgit2-devel, rpm-devel) so there should be a very low barrier for entry for any environment that would need it. > Don't get me wrong, I am not opposed to this proposal per-se. But as far > as I recall, many people were pretty upset about modular packages being > effectively only buildable in Fedora's infra and nowhere else. And I'd > very much like not to repeat this. > > > While having an option to use rpmbuild directly to build srpm/rpm from > > a dist-git repo is nice, I would say that fedpkg or mock are the main > > interfaces to do this. > > I know this answer won't satisfy everyone. > > Indeed. I think there *should* be at least a way how to produce a srpm > that can be rebuild *without* having access to Koji, mock and fedpkg > (ideally by our own infrastructure). Well, that excludes lots of options already :). One can also use preproc-rpmspec tool to get a rendered spec file (this is what mock uses). $ preproc-rpmspec pkg.spec.rpkg # prints rendered spec to stdout, pkg.spec.rpkg is a spec template > > > Cheers, > > Dan ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
On Sat, 19 Dec 2020 at 03:02, Luya Tshimbalanga wrote: > > > On 2020-12-18 11:13 a.m., Robbie Harwood wrote: > > clime writes: > > > >> On Fri, 18 Dec 2020 at 18:20, Robbie Harwood wrote: > >>> Robert-André Mauchin writes: > >>> > >>>> On 12/18/20 3:52 PM, James Szinger wrote: > >>>>> No. One can also download the sources from upstream using spectool or > >>>>> similar, even wget or curl. My local work flow is typically get or > >>>>> create spec file and patches, spectool -g, rpmbuild -bs, mock. > >>>>> > >>>> Unrelated to the topic at hand, but why do people still use rpmbuild -bs > >>>> instead of using a fedpkg mockbuild? You get a clean environment to > >>>> build and you don't have to install tons of devel packages on your > >>>> system. > >>> For me it's speed. Yes, mock gives a clean environment, but I'd rather > >>> not use it if I don't have to: the tradeoff is I don't have to *wait* > >>> for the mock to go get the tons of devel packages (and generally for > >>> repo/dnf slowness) - they're already installed on my system. > >> But you have fedpkg installed, right? I think fedpkg srpm should do a > >> good job as well. > > Probably, but that wasn't the question - the question was about > > mockbuild. > > > > Thanks, > > --Robbie > Mockbuild stores packages in /var/lib/mock and will skip those already > downloaded. You can configure dnf (slowness is a illusion because the > package manager does many task as verify the security and possible > update packages) to only use cache when needed at the cost of getting > outdated packages. Make sure to select the fastest mirror for effectiveness Well, I also feel dnf is very slow, especially on my current connection now, it takes about 10 minutes to spawn a new container and install a package. It should be much faster once it doesn't download filelists.xml unless needed. > > -- > > Luya Tshimbalanga > Fedora Design Team > Fedora Design Suite maintainer > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
On Fri, 18 Dec 2020 at 18:20, Robbie Harwood wrote: > > Robert-André Mauchin writes: > > > On 12/18/20 3:52 PM, James Szinger wrote: > >> > >> No. One can also download the sources from upstream using spectool or > >> similar, even wget or curl. My local work flow is typically get or > >> create spec file and patches, spectool -g, rpmbuild -bs, mock. > >> > > > > Unrelated to the topic at hand, but why do people still use rpmbuild -bs > > instead of using a fedpkg mockbuild? You get a clean environment to > > build and you don't have to install tons of devel packages on your system. > > For me it's speed. Yes, mock gives a clean environment, but I'd rather > not use it if I don't have to: the tradeoff is I don't have to *wait* > for the mock to go get the tons of devel packages (and generally for > repo/dnf slowness) - they're already installed on my system. But you have fedpkg installed, right? I think fedpkg srpm should do a good job as well. > > Thanks, > --Robbie > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
On Fri, 18 Dec 2020 at 15:53, James Szinger wrote: > > On Fri, 18 Dec 2020 03:04:01 +0100 > clime wrote: > > > I wouldn't call it "deprecating rpmbuild". That's certainly not at all > > my intention. > > > > As a side-point, I think the cases where bare rpmbuild is used to > > build an rpm/srpm from a dist-git repo are rather limited because you > > probably need to first download sources from lookaside cache so you > > probably need fedpkg/rpkg/centpkg/rfpkg or a similar dedicated tool. > > These tools then offer the `srpm` and `local` commands so It would > > make sense to rather use these commands or mock for subsequent > > srpm/rpm building. > > No. One can also download the sources from upstream using spectool or > similar, even wget or curl. My local work flow is typically get or > create spec file and patches, spectool -g, rpmbuild -bs, mock. > > I could not find simple instructions for the *pkg tools the last time > I looked. It should be just `fedpkg srpm` for a Fedora package. Or `rpkg srpm` with rpkg-util tool. spectool could contain native support for preprocessing as well, I will look at opening a PR for it. I think with rpmbuild something like: `rpmbuild --define "%_sourcedir $PWD" -bs file.spec` is needed, right? There is also yum/dnf builddep command that I wanted to look at. > > Jim > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
On Fri, 18 Dec 2020 at 16:23, James Szinger wrote: > > On Fri, 18 Dec 2020 00:51:49 +0100 > clime wrote: > > > Well, the users here are still packagers here no? I thought the "User" > > in the title means "end user" who shouldn't be affected by it. Maybe > > Ben can clarify this. > > I am making a distinction between Fedora packagers who use the Fedora > infrastructure to build RPMs for the Fedora repositories, and second > and third party packagers who use their own infrastructure. External > packagers count as ‘users’ for the purposes of change proposals, > especially infrastructure changes such as this. Many change proposals > have no effect for external packagers, or have the same effect as for > the general user base. This proposal, however, seems potentially > disruptive for external packagers, and I want to see those issues > specifically addressed in the change proposal. > > I will advocate that external packagers are of strategic importance to > the Fedora Project. The software they provide encourages the adoption > of Fedora. They form a pool a potential Fedora packagers. They are > the technical experts that support the local users. They are > fundamental to the Fedora Mission: > > Fedora creates an innovative platform for hardware, clouds, and > containers that enables software developers and community members > to build tailored solutions for their users. > > Supporting external packagers drives the Four Foundations: Freedom, > Friends, Features, and First. I agree the change proposal should address those issues. > > Jim > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
Dne pá 18. 12. 2020 10:52 dop. uživatel Miro Hrončok napsal: > On 12/18/20 1:19 AM, clime wrote: > >> I'd very much like to understand the impact of this on the following: > >> > >> > >> 1) Provenpackagers doing mass spec changes/updates. > > If the mass spec change/update doesn't involve an rpkg macro, then > > there is no difference. > > I don't understand how there is no difference. The spec "bare" spec file > is no > longer there. How do I parse it? How do I grep it? How do I sed it? > But the spec template is there. And the template can be modified. You can grep it and sed it, you cannot directly parse it with rpm without preprocessing it first. > >> 2) Provenpackagers and/or RelEngs doing (targeted) mass rebuilds. > > There should be no impact here. If the source git repo stays the same, > > then the same srpm as for a previous build will be produced. > > I don't understand you answer. What does this have to do with "source git > repo > stays the same" and "same srpm"? The release must be bumped, they are not > the same. > Yes, you are right. I was thinking along the lines of a feature that doesn't exist, which is inserting buildID into built rpms (not srpms) and making that an effective part of the full resulting rpm name. But there is nothing like that. Sorry for that. Release needs to be bumped even for soname-bump rebuilds (at least nowadays) so there is an impact if the target package uses git_dir_release preprocessing macro. In that case, bumping is done either by creating a new commit or a new annotated tag (the new tag will also add a new changelog entry and will make the release look nicer). Basically, if releng/ProvenPackager does a rebuild of a package manually, the recommended action currently for such a package is to add a new annotated tag (`fedpkg tag`) and write the reason for the rebuild. If a script is used, that script can be modified to automatically recognize if the target package uses the rpkg macro or not and do the required action for bumping accordingly. This is kind of a logic that should land in rpmdev-bumpspec. If this seems inconvenient, we can further discuss the options. We could e.g. make it so that just creating a new empty commit would be enough instead of tagging. > Thanks for the other answers. > > -- > Miro Hrončok > -- > Phone: +420777974800 > IRC: mhroncok > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
On Friday, 18 December 2020, Tom Stellard wrote: > On 12/17/20 11:05 AM, Ben Cotton wrote: > >> https://fedoraproject.org/wiki/Changes/Enable_Spec_File_Preprocessing >> >> >> == Summary == >> This change should enable an opt-in spec file preprocessor in Fedora >> infrastructure for the benefit of packagers. The preprocessor allows >> some very neat tricks that were impossible before, for example >> generate changelog and release automatically from git metadata or pack >> the entire dist-git repository into an rpm-source tarball (effectively >> allowing unpacked repos to live in DistGit). >> >> == Owner == >> * Name: [[User:clime| Michal Novotný]] >> * Email: cl...@fedoraproject.org >> >> >> == Detailed Description == >> >> There is a recently added feature into mock: >> [https://github.com/rpm-software-management/mock/wiki/Plugin >> -rpkg-preprocessor >> the rpkg preprocessor] which, if enabled, introduces an intermediate >> step just before srpm building. This step consists of running the spec >> file through a text preprocessing engine that includes an already >> present library of macros designed specifically for rpm spec file >> generation from git metadata. This library is called >> [https://docs.pagure.org/rpkg-util/v3/macro_reference.html >> rpkg-macros]. The macros there allow packagers to have their >> `%changelog`, `Release`, `Version`, `VCS` tag, or even `Source` fields >> automatically generated from dist-git repository data and metadata. >> The library can be easily extended in future to support more packager >> use-cases or even a completely new library can be developed that >> doesn't look at git metadata at all and instead, for example, analyses >> already present tarball content to render spec file based on upstream >> information. This doesn't mean it will happen but the framework is >> generic enough to support that. There is also support for user-defined >> macros that are loaded on-demand from a file placed alongside the >> package sources, maintained by packager. This feature wouldn't be >> enabled by this change from start but it's an example of freedom that >> the preprocessing framework is able to provide. Enabling this change >> should be very easy, basically adding: >> >> >> config_opts['plugin_conf']['rpkg_preprocessor_enable'] = True >> >> >> into mock configuration of Koji builders and using at least mock 2.7. >> Some very minor change may be also needed in Koji regarding the spec >> file lookup. >> >> Even if the change is enabled on the infrastructure level like this, >> the packager will still need to opt-in to use the preprocessor. The >> opting-in is done by placing `rpkg.conf` file into the package >> top-level directory with the following content: >> >> >> [rpkg] >> preprocess_spec = True >> >> >> When this is done by a packager, the preprocessor will be finally >> enabled for the given package. >> >> Alongside, there is an ongoing work to add the preprocessor support >> into the `rpkg` python library so that a packager can easily work with >> the spec files containing the preprocessor (rpkg) macros: >> https://pagure.io/rpkg/pull-request/530 >> >> > Is this pull request needed so that the preprocessor will run if I do > `fedpkg mockbuild` or does fedpkg already do this if the preprocessor is > enabled? The pull request is needed for it. fedpkg doesn't currently have support for preprocessing. Cheers clime > > -Tom > > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://docs.fedoraproject.org > /en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.or > g/archives/list/devel@lists.fedoraproject.org > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
On Fri, 18 Dec 2020 at 02:27, Japheth Cleaver wrote: > > On 12/17/2020 3:59 PM, Matthew Miller wrote: > > On Fri, Dec 18, 2020 at 12:51:49AM +0100, clime wrote: > > This change proposal does affect users. The User Experience section > needs to answer the following: > > Well, the users here are still packagers here no? I thought the "User" > in the title means "end user" who shouldn't be affected by it. Maybe > Ben can clarify this. > > It _is_ meant to refer to end users, but we have a lot of highly technical > end users and sysadmins who might want to download and build source RPMs. So > the answers to these questions seem like reasonable things to add. > > > They can use mock if the preprocessing will be enabled for the > respective chroots where it is enabled in Koji/Fedora. > They can't directly use rpmbuild for those packages that contain the > macros. But they can use rpkg/fedpkg to do the work. > Or preprocess spec first and then use rpmbuild. I am aware this is a > negative point of this change. > While having an option to use rpmbuild directly to build srpm/rpm from > a dist-git repo is nice, I would say that fedpkg or mock are the main > interfaces to do this. > I know this answer won't satisfy everyone. > > > Not to mention the many folks who use Fedora .src.rpms as a starting point > for EL-derivatives, or other RPM-based distros. Every time a Rawhide (or > Fedora) SRPM fails to compile because of a non-backwards-compatible change, > another frustrated sysadmin sheds a single tear. > > Deprecating rpmbuild is a major change. I wouldn't call it "deprecating rpmbuild". That's certainly not at all my intention. As a side-point, I think the cases where bare rpmbuild is used to build an rpm/srpm from a dist-git repo are rather limited because you probably need to first download sources from lookaside cache so you probably need fedpkg/rpkg/centpkg/rfpkg or a similar dedicated tool. These tools then offer the `srpm` and `local` commands so It would make sense to rather use these commands or mock for subsequent srpm/rpm building. > > -jc > > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
On Fri, 18 Dec 2020 at 00:58, Matthew Miller wrote: > > On Fri, Dec 18, 2020 at 12:24:10AM +0100, clime wrote: > > It would be possible to specify the spec template as an rpm Source so > > it would get included into the resulting srpm as well. > > Yeah I was thinking the spec file templating system could automatically add > the original spec as Source where N is one higher than the highest > existing source file. I mean something like that could be done but I need to avoid parsing spec file from within the rpkg macros because when the rpkg macros are being evaluated, there is not-yet a valid rpm spec file to be parsed. That means I cannot get the highest used Source number in the context where I would like to get it...but I think this is just one line that doesn't really need to be dynamically generated because it shouldn't change. > > > -- > Matthew Miller > > Fedora Project Leader > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
On Thu, 17 Dec 2020 at 21:23, Miro Hrončok wrote: > > On 12/17/20 8:05 PM, Ben Cotton wrote: > > https://fedoraproject.org/wiki/Changes/Enable_Spec_File_Preprocessing > > > > > > == Summary == > > This change should enable an opt-in spec file preprocessor in Fedora > > infrastructure for the benefit of packagers. The preprocessor allows > > some very neat tricks that were impossible before, for example > > generate changelog and release automatically from git metadata or pack > > the entire dist-git repository into an rpm-source tarball (effectively > > allowing unpacked repos to live in DistGit). > ... > > I'd very much like to understand the impact of this on the following: > > > 1) Provenpackagers doing mass spec changes/updates. If the mass spec change/update doesn't involve an rpkg macro, then there is no difference. If it does involve an rpkg macro, then we should directly change the macro so that the spec file gets rendered correctly according to the latest state-of-art. That probably means notifying affected packagers first but for a proven packager it is less work. There is also an option that a packager would specify the macros version with which to evaluate the spec file in `rpkg.conf` file. In that case, he would need to bump that version first so that the updated macro gets used. Not sure if something like this would be needed but this would prevent any changes in spec file unless the packager wants them on his/her own. > > 2) Provenpackagers and/or RelEngs doing (targeted) mass rebuilds. There should be no impact here. If the source git repo stays the same, then the same srpm as for a previous build will be produced. > > 3) Packagers doing `fedpkg local` builds. This PR makes sure `fedpkg local` will work: https://pagure.io/rpkg/pull-request/530 if preprocessing is enabled. There is a bit of additional work to open a PR for fedpkg to parse rpkg.conf file if it is present in a dist-git repo and enable preprocessing if it is enabled there in the file. It's just a few lines that I plan to write when I get feedback on the `rpkg` pull request. > > 5) Our downstreams rebuilding from dist-git. If they use mock or fedpkg, there should be no impact. If they use bare rpmbuild, it will no longer work and some changes will be needed. > > 4) Packages needed to be installed in buildSRPMFromSCM mock and/or Koji host. I am not sure if I understand correctly here so maybe this will need some more explanation. But the preprocessing needs some additional tooling to get installed that I tried to minimize. Basically: preproc, rpkg-macros, rpm-git-tag-sort, libgit2, git-core are the main packages needed to run the preprocessing. These packages get installed into the target chroot where the srpm is also built afterwards. They are installed only if preprocessing is enabled. > > > I'd also like to know, where exactly is the spec file pre-processed. Is it in > the buildSRPMFromSCM mock, or on the Koji host? It is preprocessed in the target chroot, i.e. in the same environment where rpmbuild -bs is called afterwards. > > > It feels like this will open a can of worms and I don't think the benefits are > worth it. IMHO we should strive to make RPM specs more flexible instead of > adding another layer on top of it. But I admit that I don't have all the > information yet. I think something like this is needed whether it is in rpm or in rpkg/mock. I think having implemented it on an upper layer than rpm is not such a huge deal. > > -- > Miro Hrončok > -- > Phone: +420777974800 > IRC: mhroncok > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
On Thu, 17 Dec 2020 at 22:04, James Szinger wrote: > > On Thu, 17 Dec 2020 14:05:40 -0500 > Ben Cotton wrote: > > > https://fedoraproject.org/wiki/Changes/Enable_Spec_File_Preprocessing > > > > == Summary == > > This change should enable an opt-in spec file preprocessor in Fedora > > infrastructure for the benefit of packagers. The preprocessor allows > > some very neat tricks that were impossible before, for example > > generate changelog and release automatically from git metadata or pack > > the entire dist-git repository into an rpm-source tarball (effectively > > allowing unpacked repos to live in DistGit). > > > > == User Experience == > > This change is intended for packagers. It should help to make a bit of > > their work easier and offer them some new interesting options. > > This change proposal does affect users. The User Experience section > needs to answer the following: Well, the users here are still packagers here no? I thought the "User" in the title means "end user" who shouldn't be affected by it. Maybe Ben can clarify this. > > 1. How does this affect users who download, maybe modify, and rebuild > the SRPM? Can they continue to use rpmbuid and mock as they have > been? Does the SRPM contain the pre-processed or post-processed spec > file? They can use mock if the preprocessing will be enabled for the respective chroots where it is enabled in Koji/Fedora. They can't directly use rpmbuild for those packages that contain the macros. But they can use rpkg/fedpkg to do the work. Or preprocess spec first and then use rpmbuild. I am aware this is a negative point of this change. While having an option to use rpmbuild directly to build srpm/rpm from a dist-git repo is nice, I would say that fedpkg or mock are the main interfaces to do this. I know this answer won't satisfy everyone. > > 2. How does this affect users who download the spec file from > src.fedoraproject.org? Do they have the tools to build the RPM? How > much harder is it? The tools will be available. It should be no-work if a person uses fedpkg or mock. Otherwise, if they really use bare rpmbuild, they will need to modify their scripts to use fedpkg/mock. Or they can use preproc first to render the template and then pass it to rpmbuild. I also planned to do a simple wrapper called '"preproc-rpmbuild" which would do that while otherwise providing the same command-line interface as rpmbuild does. > > Please remember that this is Free Software and the spec files are > useful for a broader audience than just the Fedora packagers. Right, we also use the (CentOS) spec files in a company where I currently work at. > > Jim > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
On Thu, 17 Dec 2020 at 21:34, James Cassell wrote: > > > On Thu, Dec 17, 2020, at 2:05 PM, Ben Cotton wrote: > > https://fedoraproject.org/wiki/Changes/Enable_Spec_File_Preprocessing > > > > > > == Summary == > > This change should enable an opt-in spec file preprocessor in Fedora > > infrastructure for the benefit of packagers. The preprocessor allows > > some very neat tricks that were impossible before, for example > > generate changelog and release automatically from git metadata or pack > > the entire dist-git repository into an rpm-source tarball (effectively > > allowing unpacked repos to live in DistGit). > > > > I'm skeptical. If it does pass, I'd insist on having the non-processed spec > and any required supporting files in the SRPM. It would be possible to specify the spec template as an rpm Source so it would get included into the resulting srpm as well. > > Does this relate in any way to the magic done by rdopkg dist-git <-> > source-git translation? Their approach seems very good to me, but might not > exactly overlap here. There is some overlap in the goal (allow people to work with upstream sources when convenient) but the method to achieve it is slightly different. The approach suggested here is more declarative while the rdopkg's approach is more imperative. Basically, with rdopkg, you do the upstream<->downstream conversion on the client and then just push the results to the server. With the preprocessing engine, you define the conversion in a spec file and then let the infrastructure (i.e. builders) do the work just before srpm is built. We can go into more details. There probably is some inaccuracy in what I have just said (but tried my best to explain the difference as I see it). > > V/r, > James Cassell > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
...snip... > > I'm generally not excited about this, as it adds a huge layer of > indirection and a ton of extra magic that makes it harder to decipher > what is happening. > I don't think there is any magic in it. Everything is clearly documented and every expansion clearly defined and also intuitive. If something changes spec files "behind my back" - that I would call magic but If a packager explicitly states: "Here, in this place in the spec file, I want this particular dynamic snippet to be expanded", then I think that is very transparent. Especially, if some other person looks at the spec file, he/she will know something will happen with the spec file and also what it will be (perhaps after quickly checking the docs). > > -- > 真実はいつも一つ！/ Always, there's only one truth! > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 34 Change: Enable spec file preprocessing (System-Wide Change proposal)
packed in a tarball but they are directly present in a dist-git repo).
To remind us of our collective knowledge, there can be multiple
variants of this. One of them is that the spec file is placed next to
the unpacked upstream sources (the simplest one). Then there are
approaches that try to keep the upstream (unpacked) sources and the
spec file separate - i.e. they can be in different branches, different
repos, different subdirectories - each of these use-cases can be
supported by the preprocessing engine.
But I would say this is an advanced feature of the macros and it can
be forbidden to use from the start (effectively git_dir_pack and
git_dir_archive macros wouldn't be allowed). But I will be happy to
discuss it.
It is related to source-git although the meaning is very slightly
different with respect to the separation of upstream and the
respective downstream sources - in source-git, they, by default, live
in completely different source forges and some synchronization between
the two is needed. Here, when I say "unpacked repo" I mean a repo with
mirrored upstream sources that live directly in our dist-git.
>
> > == Scope ==
> > * Proposal owners:
> > For the very basic support, probably a small patch in Koji is needed
> > to be able to lookup not only `.spec` files but also `.spec.rpkg`
> > files (the `.spec.rpkg` extension explicitly states that the spec file
> > is a template). Also the `rpmdevtools/rpmdev-bumpspec` script should
> > be tweaked to be compatible with spec files using the macros.
>
> Is the Change owner going to submit patches for fedpkg and rpmdevtools?
Yes, he will unless a rpmdevtools devel won't be faster :).
>
> > * Other developers:
> > Some optional help with `rpmdevtools/rpmdev-bumpspec` changes would be
> > welcome.
>
> rpmdev-bumpspec is a tangled mess of spaghetti code and I'd rather not
> touch it or make it even more complicated.
> I also think this can't be optional. releng uses rpmdev-bumpspec as
> part of scripted mass rebuilds, so it must work with all packages.
I meant "optional" in sense that I would optionally welcome some help
here. I can probably do it on my own, i.e. open a PR and continue from
there but I also feel like it would be great if this was more of a
collective effort.
>
> > * Release engineering: [https://pagure.io/releng/issue/9910 #9910] (a
> > check of an impact with Release Engineering is needed)
> > Enabling the rpkg_preprocessor plugin in mock config for Koji builders.
> >
> > * Policies and guidelines:
> > The new macro support should be mentioned or even described in the
> > packaging guidelines. We should decide if the full power of the
> > rpkg-macros library should be allowed from start (i.e. even unpacked
> > repos).
> >
> > * Trademark approval: N/A (not needed for this Change)
> >
> > * Alignment with Objectives: N/A
> >
> > == Upgrade/compatibility impact ==
> > Because of the opt-in nature on packager side, there should be no
> > compatibility issues.
> >
> > == How To Test ==
> > Once the feature is enabled, one can test it by providing the
> > `rpkg.conf` file with the required content in a package repository and
> > use some rpkg macro in the spec file: e.g.
> >
> >
> > Name: {{{ git_dir_name }}}
> >
> >
> > to generate the name of the package from the repository name (this
> > should actually produce the original text as package names should be
> > the same as the repository basenames).
>
> Not sure I understand, but what's the benefit of making the Name a
> macro determined by the repository name?
> As stated, they must always be the same anyway, so why make this a
> dynamic templated value?
There is not much value in this particular usage. I just tried to
recommend a way to test it out with the least amount of effort.
>
> > To try it out first without committing to dist-git, one can use `rpkg`
> > command-line tool from
> > https://copr.fedorainfracloud.org/coprs/clime/rpkg-util/ or even
> > fedpkg's koji scratch build after
> > [https://pagure.io/rpkg/pull-request/530 the work in the pyrpkg]
> > library is finished.
> >
> > One can also currently use Copr's SCM "rpkg" build method where the
> > macros are enabled but the rpkg-macros there are in version 2 whereas
> > this change is about introducing the
> > [https://docs.pagure.org/rpkg-util/v3/macro_reference.html version 3
> > rpkg-macros]. However, while there are some differences between v2 and
> > v3, the idea and most of the working is the same.
> >
> > == User Experience ==
> > This change is intended for packagers. It should help to make a bit of
> >
Re: Depend on git-core instead of git if possible
On Tue, 3 Nov 2020 at 21:22, clime wrote:
>
> On Tue, 3 Nov 2020 at 19:25, Neal Gompa wrote:
> >
> > On Tue, Nov 3, 2020 at 1:08 PM clime wrote:
> > >
> > > On Tue, 3 Nov 2020 at 17:42, Florian Weimer wrote:
> > > >
> > > > >> Or switch to depend on `%{_bindir}/git`?
> > > > >
> > > > > If we do it like this, we will never be able drop repo download times
> > > > > for Fedora users.
> > > >
> > > > Files in %{_bindir} already end up in the primary metadata, don't they?
> > >
> > > Ok, I didn't know that. Do you happen to know if there is
> > > documentation of what ends up in primary metadata and what not?
> > >
> >
> > There is not, but it's generally supposed to be */bin and */lib.
>
> I think it might be a good idea to state this in packaging guidelines...
>
> i.e. "Packages should either specify their requirements explicitly by
> package names or optionally by a filesystem path for files under
> /usr/bin and /usr/lib paths. Requirements on files at other paths are
> technically also possible but they might trigger the need for download
> of additional repodata files by dnf when such a package is being
> installed, therefore they are not recommended."
>
> ... or something like that. I will need to check the exact paths etc.
> but after some more polishcould i open a pull request for this
> somewhere? I think it doesn't matter that lazy loading is not
> implemented yet.
Well, i should have read what Vit recommended first :):
https://docs.fedoraproject.org/en-US/packaging-guidelines/#_file_and_directory_dependencies
Packages SHOULD NOT include file dependencies outside of the following
directories:
/usr/bin
/usr/sbin
/etc
So I guess...nevermind...
clime
>
> Thank you
> clime
>
> >
> >
> >
> >
> >
> > --
> > 真実はいつも一つ！/ Always, there's only one truth!
> > ___
> > devel mailing list -- devel@lists.fedoraproject.org
> > To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Depend on git-core instead of git if possible
On Tue, 3 Nov 2020 at 19:25, Neal Gompa wrote:
>
> On Tue, Nov 3, 2020 at 1:08 PM clime wrote:
> >
> > On Tue, 3 Nov 2020 at 17:42, Florian Weimer wrote:
> > >
> > > >> Or switch to depend on `%{_bindir}/git`?
> > > >
> > > > If we do it like this, we will never be able drop repo download times
> > > > for Fedora users.
> > >
> > > Files in %{_bindir} already end up in the primary metadata, don't they?
> >
> > Ok, I didn't know that. Do you happen to know if there is
> > documentation of what ends up in primary metadata and what not?
> >
>
> There is not, but it's generally supposed to be */bin and */lib.
I think it might be a good idea to state this in packaging guidelines...
i.e. "Packages should either specify their requirements explicitly by
package names or optionally by a filesystem path for files under
/usr/bin and /usr/lib paths. Requirements on files at other paths are
technically also possible but they might trigger the need for download
of additional repodata files by dnf when such a package is being
installed, therefore they are not recommended."
... or something like that. I will need to check the exact paths etc.
but after some more polishcould i open a pull request for this
somewhere? I think it doesn't matter that lazy loading is not
implemented yet.
Thank you
clime
>
>
>
>
>
> --
> 真実はいつも一つ！/ Always, there's only one truth!
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Depend on git-core instead of git if possible
On Tue, 3 Nov 2020 at 17:42, Florian Weimer wrote:
>
> >> Or switch to depend on `%{_bindir}/git`?
> >
> > If we do it like this, we will never be able drop repo download times
> > for Fedora users.
>
> Files in %{_bindir} already end up in the primary metadata, don't they?
Ok, I didn't know that. Do you happen to know if there is
documentation of what ends up in primary metadata and what not?
Thank you!
clime
> Is this about removing them? They are just 36,217 lines out of
> 2,253,333, or something like that. They also should compress well
> (better than all those hashes), so I don't see how they would impact
> metadata download times.
>
> Thanks,
> Florian
> --
> Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
> Commercial register: Amtsgericht Muenchen, HRB 153243,
> Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael
> O'Neill
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Depend on git-core instead of git if possible
On Tue, 3 Nov 2020 at 15:40, Vít Ondruch wrote:
>
>
> Dne 03. 11. 20 v 11:54 Petr Pisar napsal(a):
>
> On Mon, Nov 02, 2020 at 12:03:12PM +0100, Miro Hrončok wrote:
>
> git (amahdal, besser82, chrisw, pcahyna, pstodulk, skisela, tmz)
> git.src requires cvs
> git-cvs.noarch requires cvs
>
> ( )
> requires git
>
> Note that if xinetd indeed goes away, your package will most likely not be
> affected, unless you actually need git-cvs.
>
> Nontheless the packagers should review their packages whether they indeed have
> to depend on the full-blown git which pulls in Perl and Python. Very probably
> they could switch to a tinier git-core:
>
>
> Or switch to depend on `%{_bindir}/git`?
If we do it like this, we will never be able drop repo download times
for Fedora users.
I personally think packages should depend on package names only even
though filelists.xml lazy loading is not yet implemented
(https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/7I6HII4Y62BYVU2DEY67VMONJKZENDJC/).
clime
>
>
> Vít
>
>
>
> $ dnf --quiet repoquery --exact --whatrequires git-core |wc -l
> 48
> $ dnf --quiet repoquery --exact --whatrequires git |wc -l
> 91
>
> -- Petr
>
>
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: making download of filelists.xml(.zck) trigger on demand
On Mon, 2 Nov 2020 at 21:38, Daniel Mach wrote: > > Lazy file list loading? Yes, that's on DNF's TODO list already, but (to > be honest) not on top - there's always something more important. It's > not getting into DNF4, but it may get into DNF5 later on. Ok, good to know. Best regards clime > > > Dne 02. 11. 20 v 20:13 clime napsal(a): > > On Mon, 2 Nov 2020 at 18:55, Vít Ondruch wrote: > >> > >> > >> Dne 01. 11. 20 v 11:58 clime napsal(a): > >>> Hello! > >>> > >>> First of all, I don't really know what I am talking about here but I > >>> noticed the `dnf update` operation downloads among other things > >>> `filelists.xml` (optionally compressed by zchunk, thanks Jonathan > >>> Dieter!!!) and I remember there was a thread on devel mailing quite > >>> some time ago by Zbyszek from which I understood this data is only > >>> needed when filesystem paths are used as package requirements or when > >>> a fs path is specified as direct argument to `dnf install`. > >> > >> > >> This was the case for Yum, but have never the case for DNF since its > >> beginning. I think there were some reasons, such as that the solver in > >> case it finds it needs them would need to stop, download the data and > >> start from beginning. Other reason could be that nobody optimized it yet. > >> > >> Anyway, I guess you'll find more info in Bugzilla. > > > > Hello vit, thank you very much, I will try to look it up. M. > > > >> > >> > >> V. > >> > >> > >>>Could we > >>> then, please, trigger download of this file only if and when needed > >>> and not sooner? It seems to take more than half of the total download > >>> size e.g. for fedora-32-updates repo so it could improve repodata > >>> download times quite significantly. Again, I don't really know what I > >>> am talking about here but this came to my mind recently so I wanted to > >>> write it down just in case it would be possible :). > >>> > >>> Thank you very much > >>> clime > >>> ___ > >>> devel mailing list -- devel@lists.fedoraproject.org > >>> To unsubscribe send an email to devel-le...@lists.fedoraproject.org > >>> Fedora Code of Conduct: > >>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > >>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > >>> List Archives: > >>> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > >> ___ > >> devel mailing list -- devel@lists.fedoraproject.org > >> To unsubscribe send an email to devel-le...@lists.fedoraproject.org > >> Fedora Code of Conduct: > >> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > >> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > >> List Archives: > >> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > > ___ > > devel mailing list -- devel@lists.fedoraproject.org > > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > > Fedora Code of Conduct: > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > > List Archives: > > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > > > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: making download of filelists.xml(.zck) trigger on demand
On Mon, 2 Nov 2020 at 18:55, Vít Ondruch wrote: > > > Dne 01. 11. 20 v 11:58 clime napsal(a): > > Hello! > > > > First of all, I don't really know what I am talking about here but I > > noticed the `dnf update` operation downloads among other things > > `filelists.xml` (optionally compressed by zchunk, thanks Jonathan > > Dieter!!!) and I remember there was a thread on devel mailing quite > > some time ago by Zbyszek from which I understood this data is only > > needed when filesystem paths are used as package requirements or when > > a fs path is specified as direct argument to `dnf install`. > > > This was the case for Yum, but have never the case for DNF since its > beginning. I think there were some reasons, such as that the solver in > case it finds it needs them would need to stop, download the data and > start from beginning. Other reason could be that nobody optimized it yet. > > Anyway, I guess you'll find more info in Bugzilla. Hello vit, thank you very much, I will try to look it up. M. > > > V. > > > > Could we > > then, please, trigger download of this file only if and when needed > > and not sooner? It seems to take more than half of the total download > > size e.g. for fedora-32-updates repo so it could improve repodata > > download times quite significantly. Again, I don't really know what I > > am talking about here but this came to my mind recently so I wanted to > > write it down just in case it would be possible :). > > > > Thank you very much > > clime > > ___ > > devel mailing list -- devel@lists.fedoraproject.org > > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > > Fedora Code of Conduct: > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > > List Archives: > > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
making download of filelists.xml(.zck) trigger on demand
Hello! First of all, I don't really know what I am talking about here but I noticed the `dnf update` operation downloads among other things `filelists.xml` (optionally compressed by zchunk, thanks Jonathan Dieter!!!) and I remember there was a thread on devel mailing quite some time ago by Zbyszek from which I understood this data is only needed when filesystem paths are used as package requirements or when a fs path is specified as direct argument to `dnf install`. Could we then, please, trigger download of this file only if and when needed and not sooner? It seems to take more than half of the total download size e.g. for fedora-32-updates repo so it could improve repodata download times quite significantly. Again, I don't really know what I am talking about here but this came to my mind recently so I wanted to write it down just in case it would be possible :). Thank you very much clime ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: our containers with alias vim=vi
On Tue, 13 Oct 2020 at 16:14, Zdenek Dohnal wrote:
>
> On 10/13/20 12:34 PM, Jonathan Wakely wrote:
> > On 13/10/20 07:45 +0200, Zdenek Dohnal wrote:
> >>
> >> On 10/12/20 5:15 PM, Joe Doss wrote:
> >>> On 10/12/20 1:50 AM, Zdenek Dohnal wrote:
> >>>> This would break using Vim when vim-minimal and vim-enhanced are
> >>>> installed (it would start Vi instead of typed Vim). To make it work,
> >>>> vim-minimal would have to conflict with vim-enhanced, which doesn't
> >>>> make
> >>>> sense - Vi and Vim binaries can exist together just fine.
> >>>
> >>> I have vim-enhanced and vim-minimal installed
> >>>
> >>> # rpm -qa |grep vim
> >>> vim-common-8.2.1770-1.fc33.x86_64
> >>> vim-filesystem-8.2.1770-1.fc33.noarch
> >>> vim-minimal-8.2.1770-1.fc33.x86_64
> >>> vim-enhanced-8.2.1770-1.fc33.x86_64
> >>>
> >>> and when I type vi it launches vim.
> >> I'm sorry I forgot about this alias - yes, there is an alias which is
> >> applied when both - vim-minimal and vim-enhanced - are installed so it
> >> launches 'vim' when you type 'vi'. I would say less people will complain
> >> if something has more features than if it has less, so I'm content with
> >> this alias.
> >>>
> >>> # whereis vi
> >>> vi: /usr/bin/vi /usr/share/man/man1p/vi.1p.gz
> >>> /usr/share/man/man1/vi.1.gz
> >>> # /usr/bin/vi --version
> >>> VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Sep 29 2020 00:00:00)
> >>>
> >>> It doesn't look like these are existing together just fine. It seems
> >>> that vim takes over vi. Shouldn't these conflict and only one can be
> >>> installed over the other?
> >> 'Vi' as the original project is no longer (I'm not sure for how long)
> >> shipped in Fedora. 'Vi' we ship is just 'VIM' compiled with 'small' set
> >> of features (f.e. without syntax highlighting) to mimic the original
> >> 'Vi'. So if you run 'vi --version' it always shows 'VIM'.
> >>>
> >>>> In the end I find it incorrect to mislead users by default by telling
> >>>> them 'Vim works' but Vi is run instead - Vi and Vim don't have the
> >>>> same
> >>>> set of features, which may lead into bug reports caused by this
> >>>> mistake.
> >>>
> >>> Isn't that the reverse behavior detailed above? I type vi on Fedora
> >>> Workstation it launches vim? I assume this isn't causing bug reports.
> >> You're right, as I wrote above - aliasing vi->vim doesn't seem as a
> >> problem to me, but aliasing vim->vi as clime suggested can cause mislead
> >> for users.
> >
> > I would also appreciate if "vim" ran the executable installed by
> > vim-minimal. I frequently type "vim" on servers I don't own and then
> > grumble that it fails and I have to run "vi" instead. It **is** vim,
> > it's just not installed with that name. Insisting that users call it
> > vi when we know it's vim and they know it's vim seems unnecessary.
>
> It's Vim but with a different set of features - VIM compiled as Vi
> binary is trying to be small, kind of simulate Vi behavior without
> setting 'compatible'.
>
> Since you know it has less features, good for you. But unfortunately,
> not all users know - there was already a report about Vim missing
> highlighting, and the reporter was running /usr/bin/vi. So my aim is to
> prevent such a report.
>
> >
> > $ rpm -qf /usr/bin/vi
> > vim-minimal-8.2.1770-1.fc32.x86_64
> >
> > Could vim-minimal and vim-enhanced both install the same
> > /etc/profile.d/vim.sh file that did something like this?
> Installing the same file would mean to set conflicts between those two
> package, wouldn't it? Or would you mind explaining how to achieve it?
> >
> > if [ -n "${BASH_VERSION-}" -o -n "${KSH_VERSION-}" -o -n
> > "${ZSH_VERSION-}" ]; then
> > [ "`/usr/bin/id -u 2>/dev/null || echo 0`" -le 200 ] && return
> > # for bash and and ksh and zsh
> > case "$(type -t vim)-$(type -t vi)" in
> > file-file)
> > # both vim and vi are present
> > alias vi=vim
> > alias view="vim -R"
> > ;;
> > -file)
> > # only vim-minimal is installed, expose it as 'vim' too
> > alias vim=vi
> > ;;
> > esac
> > fi
> >
> Looks good,
Re: our containers with alias vim=vi
On Mon, 12 Oct 2020 at 07:39, Zdenek Dohnal wrote: > > On 10/10/20 2:37 PM, clime wrote: > > Hello, > > > > could Fedora and CentOS containers for docker and podman come with > > `alias vim=vi` in ~/.bashrc? > > > > I would very much welcome it as I am used to type vim everywhere but > > if vi starts instead I am happy too. I know that the solution is to > > create a customized container but often I want to try something on > > vanilla containers from the whole range. > > IMHO it is not a good idea. Some users which don't have to know the > problem can run 'vi' while thinking they run 'vim' and be surprised that > most 'Vim' features don't work and they will file a bug tickets, which > will be irrelevant, consuming reporter's's time. well, it would be good if vim itself display in which mode it runs. So then if I run "vim", I get "vi", i will know, ok, i got only the stripped down version because i am in a container and the "extension" is not yet installed. I would very much appreciate it as a user (about 16 years) of the great vim. Usually in a vanilla container, i just want to run an editor quickly to look at a file or quickly edit something - i don't really care about user experience because if I did, I would already customized the container. I wonder if typing vi instead of "vim" on my computer has any effect. I am quite positive that it had at some point in time but not sure about nowadays. clime > > This problem should be solved by user (when he know there is no Vim and > excepts to use Vi, then he creates alias) or by installing vim-enhanced. > > -- > Zdenek Dohnal > Software Engineer > Red Hat Czech - Brno TPB-C > > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
our containers with alias vim=vi
Hello, could Fedora and CentOS containers for docker and podman come with `alias vim=vi` in ~/.bashrc? I would very much welcome it as I am used to type vim everywhere but if vi starts instead I am happy too. I know that the solution is to create a customized container but often I want to try something on vanilla containers from the whole range. Didn't want to write about this first but maybe there are more people with the same problem. clime ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: rpkg broke?
On Mon, 28 Sep 2020 at 23:03, Richard Shaw wrote: > > I was able to start a build after installing the updated package, but running > rpkg by itself still produces the same confusing output: > > $ rpkg > 'Namespace' object has no attribute 'command' Yes, thank you. I plan to have this fixed for version 3. clime > > Thanks, > Richard > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: rpkg broke?
On Mon, 28 Sep 2020 at 19:17, Richard Shaw wrote: > > On Mon, Sep 28, 2020 at 9:26 AM clime wrote: >> >> On Mon, 28 Sep 2020 at 14:44, Richard Shaw wrote: >> > >> > I haven't seen anything obvious in the mailing list lately so maybe it's >> > just me? >> > >> > $ rpkg >> > 'Namespace' object has no attribute 'command' >> >> Hello, >> >> What are versions of rpkg and python3-rpkg in your system? By the way, >> do you use that command for something actually? > > > $ rpm -qa | grep rpkg > python3-rpkg-1.61-1.fc32.noarch > rpkg-2.7-5.fc32.noarch > rpkg-common-1.61-1.fc32.noarch > > $ rpkg build NBEMS-testing > __init__() got an unexpected keyword argument 'kojiconfig' > > That's when I tried running it by itself to see what would happen. Ah, okay, that was a problem with recent changes to the python3-rpkg library. I have pushed an update fixing this today (https://bodhi.fedoraproject.org/updates/FEDORA-2020-44457c00e8). clime > > Thanks, > Richard > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: rpkg broke?
On Mon, 28 Sep 2020 at 14:44, Richard Shaw wrote: > > I haven't seen anything obvious in the mailing list lately so maybe it's just > me? > > $ rpkg > 'Namespace' object has no attribute 'command' Hello, What are versions of rpkg and python3-rpkg in your system? By the way, do you use that command for something actually? Thank you clime > > Thanks, > Richard > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: About the Future of Communishift
On Fri, 4 Sep 2020 at 12:59, clime wrote: > > On Fri, 4 Sep 2020 at 12:48, Aoife Moloney wrote: > > > > Good Morning Everyone, > > > > I wanted to share with you some information regarding the current > > state and future of Communishift. The infrastructure team presented on > > this project back in 2019 during Nest [1] [2], and since then, we have > > deployed it, started using it and had to shut it down for > > the colo-move. > > > > As a number of people have noted, it has not come back up yet, and > > during Nest this year, we had hinted that communishift is not going to > > come back alive looking > > the same as when we shut it down, and that is unfortunately true. > > > > The idea for communishift was to give to anyone in the community a place > > where > > they could run any application they wish to provide to the community. > > This was a proper place where Joe and Jane could offer the service foo to > > the > > foo SIG without engaging the infrastructure's team responsibility to keep > > the > > service up and running. The infrastructure team would have been able to say: > > "well the openshift cluster is running, so if the app isn't, talk to the > > application maintainer, there is nothing we can do about it". > > > > Basically, it gave a place where we could experiment with new apps > > without adding too > > much work to the infrastructure team. > > > > However, the General Data Protection Regulation (GDPR) [3] and the > > California > > Consumer Privacy Act (CCPA) [4] basically makes the Fedora Infrastructure > > team > > (and thus Red Hat) responsible for the content hosted by any services > > running in > > our infrastructure. In other words, the Fedora Infrastructure team would be > > responsible to answer all GDPR/CCPA related requests and requirements for > > any > > and all services running in communishift (services that the team has 0 > > knowledge > > about, that's the whole goal of communishift). > > > > For these reasons communishift is not going to come back to life in the > > same way > > it was shut down for the colo move. > > > > We have not given up on the original idea though (ie: providing a place > > where > > community members can deploy applications without adding work on the > > infrastructure team), however, as with anything involving legal, this is > > going > > to be a slow process. We will share any information as soon as we are able. > > > > > > We're sorry for the inconvenience this causes, we really would like the > > situation to be different but we also appreciate these regulations for what > > they > > are (protecting our personal information) so we want to respect them. > > > > > > Hoping this clarifies the situation around communishift a bit. > > > > Aoife, Kevin & Pingou > > - On behalf of the Fedora Infrastructure team > > Hello Aoife, > > is it working right now so that I can deploy my community app there or > currently not working at all? Or better...is there at least some alternative or some way to deploy a community app these days? Or currently none. :) Thank you for the answer clime > > Thank you > clime > > > > > > > [1] https://fedoraproject.org/wiki/Infrastructue/Communishift > > [2] > > https://www.youtube.com/watch?v=phCHilTEQb4=PL0x39xti0_64C75dRUuwlXlfYRgjgdEP4=8=0s > > [3] https://gdpr-info.eu/ > > [4] https://www.oag.ca.gov/privacy/ccpa > > > > [4] https://www.oag.ca.gov/privacy/ccpa > > > > -- > > Aoife Moloney > > Product Owner > > Community Platform Engineering Team > > Red Hat EMEA > > Communications House > > Cork Road > > Waterford > > ___ > > devel-announce mailing list -- devel-annou...@lists.fedoraproject.org > > To unsubscribe send an email to devel-announce-le...@lists.fedoraproject.org > > Fedora Code of Conduct: > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > > List Archives: > > https://lists.fedoraproject.org/archives/list/devel-annou...@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: About the Future of Communishift
On Fri, 4 Sep 2020 at 12:48, Aoife Moloney wrote: > > Good Morning Everyone, > > I wanted to share with you some information regarding the current > state and future of Communishift. The infrastructure team presented on > this project back in 2019 during Nest [1] [2], and since then, we have > deployed it, started using it and had to shut it down for > the colo-move. > > As a number of people have noted, it has not come back up yet, and > during Nest this year, we had hinted that communishift is not going to > come back alive looking > the same as when we shut it down, and that is unfortunately true. > > The idea for communishift was to give to anyone in the community a place where > they could run any application they wish to provide to the community. > This was a proper place where Joe and Jane could offer the service foo to the > foo SIG without engaging the infrastructure's team responsibility to keep the > service up and running. The infrastructure team would have been able to say: > "well the openshift cluster is running, so if the app isn't, talk to the > application maintainer, there is nothing we can do about it". > > Basically, it gave a place where we could experiment with new apps > without adding too > much work to the infrastructure team. > > However, the General Data Protection Regulation (GDPR) [3] and the California > Consumer Privacy Act (CCPA) [4] basically makes the Fedora Infrastructure team > (and thus Red Hat) responsible for the content hosted by any services running > in > our infrastructure. In other words, the Fedora Infrastructure team would be > responsible to answer all GDPR/CCPA related requests and requirements for any > and all services running in communishift (services that the team has 0 > knowledge > about, that's the whole goal of communishift). > > For these reasons communishift is not going to come back to life in the same > way > it was shut down for the colo move. > > We have not given up on the original idea though (ie: providing a place where > community members can deploy applications without adding work on the > infrastructure team), however, as with anything involving legal, this is going > to be a slow process. We will share any information as soon as we are able. > > > We're sorry for the inconvenience this causes, we really would like the > situation to be different but we also appreciate these regulations for what > they > are (protecting our personal information) so we want to respect them. > > > Hoping this clarifies the situation around communishift a bit. > > Aoife, Kevin & Pingou > - On behalf of the Fedora Infrastructure team Hello Aoife, is it working right now so that I can deploy my community app there or currently not working at all? Thank you clime > > > [1] https://fedoraproject.org/wiki/Infrastructue/Communishift > [2] > https://www.youtube.com/watch?v=phCHilTEQb4=PL0x39xti0_64C75dRUuwlXlfYRgjgdEP4=8=0s > [3] https://gdpr-info.eu/ > [4] https://www.oag.ca.gov/privacy/ccpa > > [4] https://www.oag.ca.gov/privacy/ccpa > > -- > Aoife Moloney > Product Owner > Community Platform Engineering Team > Red Hat EMEA > Communications House > Cork Road > Waterford > ___ > devel-announce mailing list -- devel-annou...@lists.fedoraproject.org > To unsubscribe send an email to devel-announce-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel-annou...@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Review swap
Hello, I have a package rpm-git-tag-sort for a review. Can I swap it with somebody? https://bugzilla.redhat.com/show_bug.cgi?id=1858256 clime ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: RHEL 9 and modularity
On Wed, 24 Jun 2020 at 10:35, Petr Pisar wrote: > > On Wed, Jun 24, 2020 at 10:22:38AM +0200, clime wrote: > > On Wed, 24 Jun 2020 at 09:40, Petr Pisar wrote: > > > > > > On Wed, Jun 24, 2020 at 06:51:36AM +, Zbigniew Jędrzejewski-Szmek > > > wrote: > > > > Yes. Putting the "stream identification" into the package name is the > > > > most natural solution, and has been floated various times. > > > > > > This already happens. But not in Fedora. In RHEL, modular packages have > > > Modularitylabel RPM tag that carries the module name and stream. > > > > Does "ModularityLabel" actually propagates to rpm package name or is > > it just a "hidden" rpm attribute? > > > It's a RPM tag as well as a package name or a package version are the RPM > tags. I don't understand your question. Well, the original sentence was: "Putting the "stream identification" into the package *name* is the most natural solution". And the answer was: "This already happens. But not in Fedora. In RHEL, ..." But my question was answered, thank you. > > -- Petr > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: RHEL 9 and modularity
On Wed, 24 Jun 2020 at 09:40, Petr Pisar wrote: > > On Wed, Jun 24, 2020 at 06:51:36AM +, Zbigniew Jędrzejewski-Szmek wrote: > > Yes. Putting the "stream identification" into the package name is the > > most natural solution, and has been floated various times. > > This already happens. But not in Fedora. In RHEL, modular packages have > Modularitylabel RPM tag that carries the module name and stream. Does "ModularityLabel" actually propagates to rpm package name or is it just a "hidden" rpm attribute? > > -- Petr > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: RHEL 9 and modularity
On Tue, 23 Jun 2020 at 20:59, Terry Bowling wrote: > > On Tue, Jun 23, 2020 at 2:33 PM clime wrote: >> >> >> So I don't really get even after almost five years where modularity is >> going or what it wants to achieve. I don't understand its use-case for >> any of Fedora, RHEL, and CentOS because disabling >> parallel-installability to allow parallel availability is imho not >> really an option. But yeah...maybe I am missing some angle. In that >> case, please, explain it to me because I would really like to >> understand... >> >> clime > > > clime, > I just started a new thread where I try to share some background history and > context. Hopefully it helps. It should direct you to all of the other > documentation that explains most of your questions. > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/SHMBYIGHHQYMNYD4HL2LV45BVZVZ3I5B/ Great, thank you. > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: RHEL 9 and modularity
On Mon, 22 Jun 2020 at 08:14, Zbigniew Jędrzejewski-Szmek wrote: > > On Mon, Jun 22, 2020 at 04:55:10AM +0200, clime wrote: > > >> > > Hello Josh, > > >> > > > > >> > > you can change the artifact type while keeping interface the same and > > >> > > it would be a _HUGE_ win because it would make modularity finally > > >> > > understandable for mere humans and better maintainable. > > >> > > > > >> > > Namely, modules should become rpms and therefore obey standard rpm > > >> > > rules. > > >> > > > >> > I'm not sure I entirely understand what you mean, but it sounds like > > >> > you have some interesting ideas. > > >> > > > >> > I'm looking forward to seeing what you and the community can build > > >> > from them, and how they could be brought into RHEL 10+! That kind of > > >> > collaboration is what makes Fedora great. > > >> > > >> I know this probably won't change anything because this was mentioned > > >> many times (by me at least) and nothing has changed but still... > > >> > > >> Currently, modules are essentially yum sub-repos, they are not really > > >> "modules", instead they are collections of rpms that reinvent rpm-like > > >> relations (obsoletes, requires, build-requires, etc.). > > >> > > >> There is no reason for this wheel-reintervention. Modules (the > > >> collections) can be simply squashed into an rpm by automation and this > > >> resulting rpm can go to the modular repo together with other modules. > > I agree with this general idea, even if not with the exact implementation > (comments below). In the past this was stated as "divorcing the build ordering > mechanism from the rpm delivery mechanism". The fact that we have two layers > of dependencies make Modularity conceptually hard and destroy the interaction > with the dependency solver. Also, if we disconnect the build and delivery > mechanisms, we can iterate and improve both separately. > > > >> That way we don't have two types of objects we complex inter-relations > > >> but only one we well-known behavior. > > >> > > >> I wonder if this is clear to everyone but nobody really cares or > > >> doesn't really want to say it or I don't know. > > >> > > >> Is this clear to everyone? I mean either I am stating an obvious stuff > > >> that nobody really considers worth typing or idk. > > > > > > > > > How would this work when there are optional rpms in the module? > > > > > > You do not need to install every rpm in eg the php module (different > > > graphics/database backends) for that module to be useful, but every > > > version of the module will have the rpm as an option which wont work > > > outside a module of multiple rpms. > > > > Glad you ask, I wasn't precise... > > > > Well, I didn't mean everything always needs to be squashed, instead, > > it would be an optional step in modulemd processing. > > So... if it's only optional, that means that the general case where > squashing is not done needs to be solved anyway. And once you have > solved the general case, what would the point of squashing be? > Thus, I don't find squashing useful. > > > For some > > use-cases (like delicately compiled postgresql server), you can create > > a single rpm that contains all - postgresql-server, postgresql, > > postgresql-libs compiled in a specific way, optionally with some > > postgresql modules pre-included, so it would be let's say time-series > > optimized postgresql. Here it makes sense to make a single rpm from it > > - you install that and you are all set up for your use-case. > > > > Then there are language stacks where you might want to build things in > > a specific order - there nothing really needs to be squashed (or > > certain subset can if it makes sense) but you can still use modularity > > to easily batch-build certain rpms. If there are runtime optional > > deps, they can be described by Recommends/Suggests. > > > > Basically, once a "module" (things that comes from modulemd) is built, > > it should be put into normal repos and the "module" boundary should be > > forgotten (unless it is a single rpm), i.e. "module" is a built-time > > thing, at install-time we just have standard packages with standard > > deps. > > Yep. > &g
Re: RHEL 9 and modularity
On Mon, 22 Jun 2020 at 08:14, Zbigniew Jędrzejewski-Szmek wrote: > > On Mon, Jun 22, 2020 at 04:55:10AM +0200, clime wrote: > > >> > > Hello Josh, > > >> > > > > >> > > you can change the artifact type while keeping interface the same and > > >> > > it would be a _HUGE_ win because it would make modularity finally > > >> > > understandable for mere humans and better maintainable. > > >> > > > > >> > > Namely, modules should become rpms and therefore obey standard rpm > > >> > > rules. > > >> > > > >> > I'm not sure I entirely understand what you mean, but it sounds like > > >> > you have some interesting ideas. > > >> > > > >> > I'm looking forward to seeing what you and the community can build > > >> > from them, and how they could be brought into RHEL 10+! That kind of > > >> > collaboration is what makes Fedora great. > > >> > > >> I know this probably won't change anything because this was mentioned > > >> many times (by me at least) and nothing has changed but still... > > >> > > >> Currently, modules are essentially yum sub-repos, they are not really > > >> "modules", instead they are collections of rpms that reinvent rpm-like > > >> relations (obsoletes, requires, build-requires, etc.). > > >> > > >> There is no reason for this wheel-reintervention. Modules (the > > >> collections) can be simply squashed into an rpm by automation and this > > >> resulting rpm can go to the modular repo together with other modules. > > I agree with this general idea, even if not with the exact implementation > (comments below). In the past this was stated as "divorcing the build ordering > mechanism from the rpm delivery mechanism". The fact that we have two layers > of dependencies make Modularity conceptually hard and destroy the interaction > with the dependency solver. Also, if we disconnect the build and delivery > mechanisms, we can iterate and improve both separately. Indeed, I agree build and delivery mechanisms should be treated independently. Things would finally get a clear shape. > > > >> That way we don't have two types of objects we complex inter-relations > > >> but only one we well-known behavior. > > >> > > >> I wonder if this is clear to everyone but nobody really cares or > > >> doesn't really want to say it or I don't know. > > >> > > >> Is this clear to everyone? I mean either I am stating an obvious stuff > > >> that nobody really considers worth typing or idk. > > > > > > > > > How would this work when there are optional rpms in the module? > > > > > > You do not need to install every rpm in eg the php module (different > > > graphics/database backends) for that module to be useful, but every > > > version of the module will have the rpm as an option which wont work > > > outside a module of multiple rpms. > > > > Glad you ask, I wasn't precise... > > > > Well, I didn't mean everything always needs to be squashed, instead, > > it would be an optional step in modulemd processing. > > So... if it's only optional, that means that the general case where > squashing is not done needs to be solved anyway. And once you have > solved the general case, what would the point of squashing be? > Thus, I don't find squashing useful. > > > For some > > use-cases (like delicately compiled postgresql server), you can create > > a single rpm that contains all - postgresql-server, postgresql, > > postgresql-libs compiled in a specific way, optionally with some > > postgresql modules pre-included, so it would be let's say time-series > > optimized postgresql. Here it makes sense to make a single rpm from it > > - you install that and you are all set up for your use-case. > > > > Then there are language stacks where you might want to build things in > > a specific order - there nothing really needs to be squashed (or > > certain subset can if it makes sense) but you can still use modularity > > to easily batch-build certain rpms. If there are runtime optional > > deps, they can be described by Recommends/Suggests. > > > > Basically, once a "module" (things that comes from modulemd) is built, > > it should be put into normal repos and the "module" boundary should be > > forgotten (unless it is a single rpm), i.e. "module" is a built-time
Re: RHEL 9 and modularity
On Mon, 22 Jun 2020 at 04:12, Naheem Zaffar wrote: > > > > On Mon, 22 Jun 2020, 02:57 clime, wrote: >> >> On Fri, 19 Jun 2020 at 01:59, Josh Boyer wrote: >> > >> > On Thu, Jun 18, 2020 at 5:51 PM clime wrote: >> > > >> > > On Thu, 18 Jun 2020 at 15:25, Josh Boyer wrote: >> > > > >> > > > On Thu, Jun 18, 2020 at 9:05 AM Igor Raits >> > > > wrote: >> > > > > >> > > > > -BEGIN PGP SIGNED MESSAGE- >> > > > > Hash: SHA512 >> > > > > >> > > > > On Thu, 2020-06-18 at 08:44 -0400, Josh Boyer wrote: >> > >> > >> > >> > > > > > Hopefully that provides some context and helps FESCo and the wider >> > > > > > community understand where Red Hat is headed with modularity on the >> > > > > > Enterprise side. >> > > > > >> > > > > Sadly no. It helps to understand your plans, however it does not help >> > > > > to understand the reasons behind, whether you can't change UX in the >> > > > > RHEL 9, or you think that technology is good enough for your >> > > > > use-cases >> > > > > or any other reasons. >> > > > >> > > > The base requirement is that the UX remain largely the same. As I >> > > > said, from a RHEL perspective, we need RHEL 8 and RHEL 9 to have >> > > > commonality so that our customers are not forced to learn something >> > > > entirely different to adopt RHEL 9. Improvements in the underlying >> > > > functionality are of course welcome and planned, but we are not going >> > > > to do something like replace modules with a different artifact type, >> > > >> > > Hello Josh, >> > > >> > > you can change the artifact type while keeping interface the same and >> > > it would be a _HUGE_ win because it would make modularity finally >> > > understandable for mere humans and better maintainable. >> > > >> > > Namely, modules should become rpms and therefore obey standard rpm rules. >> > >> > I'm not sure I entirely understand what you mean, but it sounds like >> > you have some interesting ideas. >> > >> > I'm looking forward to seeing what you and the community can build >> > from them, and how they could be brought into RHEL 10+! That kind of >> > collaboration is what makes Fedora great. >> >> I know this probably won't change anything because this was mentioned >> many times (by me at least) and nothing has changed but still... >> >> Currently, modules are essentially yum sub-repos, they are not really >> "modules", instead they are collections of rpms that reinvent rpm-like >> relations (obsoletes, requires, build-requires, etc.). >> >> There is no reason for this wheel-reintervention. Modules (the >> collections) can be simply squashed into an rpm by automation and this >> resulting rpm can go to the modular repo together with other modules. >> >> That way we don't have two types of objects we complex inter-relations >> but only one we well-known behavior. >> >> I wonder if this is clear to everyone but nobody really cares or >> doesn't really want to say it or I don't know. >> >> Is this clear to everyone? I mean either I am stating an obvious stuff >> that nobody really considers worth typing or idk. > > > How would this work when there are optional rpms in the module? > > You do not need to install every rpm in eg the php module (different > graphics/database backends) for that module to be useful, but every version > of the module will have the rpm as an option which wont work outside a module > of multiple rpms. Glad you ask, I wasn't precise... Well, I didn't mean everything always needs to be squashed, instead, it would be an optional step in modulemd processing. For some use-cases (like delicately compiled postgresql server), you can create a single rpm that contains all - postgresql-server, postgresql, postgresql-libs compiled in a specific way, optionally with some postgresql modules pre-included, so it would be let's say time-series optimized postgresql. Here it makes sense to make a single rpm from it - you install that and you are all set up for your use-case. Then there are language stacks where you might want to build things in a specific order - there nothing really needs to be squashed (or certain subset can if it makes sense) but you can still use modularity to e
Re: RHEL 9 and modularity
On Fri, 19 Jun 2020 at 01:59, Josh Boyer wrote: > > On Thu, Jun 18, 2020 at 5:51 PM clime wrote: > > > > On Thu, 18 Jun 2020 at 15:25, Josh Boyer wrote: > > > > > > On Thu, Jun 18, 2020 at 9:05 AM Igor Raits > > > wrote: > > > > > > > > -BEGIN PGP SIGNED MESSAGE- > > > > Hash: SHA512 > > > > > > > > On Thu, 2020-06-18 at 08:44 -0400, Josh Boyer wrote: > > > > > > > > Hopefully that provides some context and helps FESCo and the wider > > > > > community understand where Red Hat is headed with modularity on the > > > > > Enterprise side. > > > > > > > > Sadly no. It helps to understand your plans, however it does not help > > > > to understand the reasons behind, whether you can't change UX in the > > > > RHEL 9, or you think that technology is good enough for your use-cases > > > > or any other reasons. > > > > > > The base requirement is that the UX remain largely the same. As I > > > said, from a RHEL perspective, we need RHEL 8 and RHEL 9 to have > > > commonality so that our customers are not forced to learn something > > > entirely different to adopt RHEL 9. Improvements in the underlying > > > functionality are of course welcome and planned, but we are not going > > > to do something like replace modules with a different artifact type, > > > > Hello Josh, > > > > you can change the artifact type while keeping interface the same and > > it would be a _HUGE_ win because it would make modularity finally > > understandable for mere humans and better maintainable. > > > > Namely, modules should become rpms and therefore obey standard rpm rules. > > I'm not sure I entirely understand what you mean, but it sounds like > you have some interesting ideas. > > I'm looking forward to seeing what you and the community can build > from them, and how they could be brought into RHEL 10+! That kind of > collaboration is what makes Fedora great. I know this probably won't change anything because this was mentioned many times (by me at least) and nothing has changed but still... Currently, modules are essentially yum sub-repos, they are not really "modules", instead they are collections of rpms that reinvent rpm-like relations (obsoletes, requires, build-requires, etc.). There is no reason for this wheel-reintervention. Modules (the collections) can be simply squashed into an rpm by automation and this resulting rpm can go to the modular repo together with other modules. That way we don't have two types of objects we complex inter-relations but only one we well-known behavior. I wonder if this is clear to everyone but nobody really cares or doesn't really want to say it or I don't know. Is this clear to everyone? I mean either I am stating an obvious stuff that nobody really considers worth typing or idk. Anyway best regards clime > > josh > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: RHEL 9 and modularity
Miller has often given a common > example of having two streams of software that can build across RHEL, > Fedora, CentOS, etc. This is value for end user consumption, and > having older software available in Fedora is a usecase modularity can > help address. However, I would prefer to avoid discussing value to > Fedora in this thread. There are so many other threads debating that > already :) > > > > Hopefully that provides some context and helps FESCo and the wider > > > community understand where Red Hat is headed with modularity on the > > > Enterprise side. > > > > Sadly no. It helps to understand your plans, however it does not help > > to understand the reasons behind, whether you can't change UX in the > > RHEL 9, or you think that technology is good enough for your use-cases > > or any other reasons. > > The base requirement is that the UX remain largely the same. As I > said, from a RHEL perspective, we need RHEL 8 and RHEL 9 to have > commonality so that our customers are not forced to learn something > entirely different to adopt RHEL 9. Improvements in the underlying > functionality are of course welcome and planned, but we are not going > to do something like replace modules with a different artifact type, Hello Josh, you can change the artifact type while keeping interface the same and it would be a _HUGE_ win because it would make modularity finally understandable for mere humans and better maintainable. Namely, modules should become rpms and therefore obey standard rpm rules. Thanks for looking at this clime > or add separate discrete repos per Application Stream, etc. > > > Basically this email just says "We decided for Modularity in RHEL 9 and > > we would like to do it in Fedora Infrastructure first". > > Mostly, yes. We were told there was ambiguity on whether modularity > would be used in RHEL 9 or not. Very clearly it will, so I wanted to > get ahead of that. > > josh > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: moreutils newly in PowerTools repo in centos-8
On Sat, 16 May 2020 at 20:33, Paul Howarth wrote: > > On Sat, 16 May 2020 20:23:35 +0200 > clime wrote: > > I am co-maintaining an epel-7 package (dist-git) and I wanted to add > > it to epel-8 too. But right now, it wouldn't install because one of > > the dependencies (moreutils) is newly in PowerTools repo > > (https://bugzilla.redhat.com/show_bug.cgi?id=1833810). > > > > I assume that basically means I cannot depend on such package from an > > epel-8 package or at least I will solve it like this by removing the > > dependency. Something like this is a bit surprising, however. I didn't > > know a repo like that existed. > > You can safely depend on things in the PowerTools repo as it's expected > that EPEL-8 users will also use that repository: > > https://fedoraproject.org/wiki/EPEL#How_can_I_use_these_extra_packages.3F > > Looking at the BZ ticket, it seems you discovered that yourself. I think in the end it is actually perl(IPC::Run) which is provided by PowerTools. The thing is that the yum error message doesn't say "Enable PowerTools to install this package". I am a long term epel user and I had no idea about PowerTools. clime > > Paul. > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
moreutils newly in PowerTools repo in centos-8
Hello, I am co-maintaining an epel-7 package (dist-git) and I wanted to add it to epel-8 too. But right now, it wouldn't install because one of the dependencies (moreutils) is newly in PowerTools repo (https://bugzilla.redhat.com/show_bug.cgi?id=1833810). I assume that basically means I cannot depend on such package from an epel-8 package or at least I will solve it like this by removing the dependency. Something like this is a bit surprising, however. I didn't know a repo like that existed. clime ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Is dist-git a good place for work?
On Thu, 14 May 2020 at 14:31, Ondřej Lysoněk wrote: > > Hunor Csomortáni writes: > > > On Wed, May 6, 2020 at 10:24 PM Simo Sorce wrote: > >> Well, a way to allow force pushes would be to have a git hook that > >> branches the tree before the force push. (creating a branch named > >> something like audit-force-push-) > >> That way you can retain data for legal/auditing reasons, while allowing > >> every day history to be rewritten. > > > > Wouldn't it be easier to approach this from a build system perspective > > and let for example the build system (or tools) tag the commits which > > were built from with some for-ever-living tags? This would still > > ensure a complete audit trail for whatever was built and shipped, but > > could eliminate the need for a complete lock down of dist/source-git. > > > >> Not sure how "nice" that would be for an auditor that has to > >> reconstruct what happened over multiple force pushes that way, it also > >> will generate quite an amount of noisy metadata (branches), but it > >> could work. > > > > Refs created for auditing purposes could be kept in a separate git > > namespace so they don't create noise in everyday workflows. > > As someone who works with git history all the time, I cannot imagine how > I would work in such an environment. Consider the simple task of finding > the commit that first introduced a downstream change. Currently with > dist-git, I can just do 'git log patch-file', scroll to the very end and > be done with it. > It's a good point that this operation would be harder but it could be solved, I think. I mean it could have beneficial features for maybe not all packages but at least some. I suspect on such scale as Fedora operates, it might be quite hard to do something which improves things for everybody. > If what you're proposing was implemented, then I'd have to manually try > all the tags until I found the "right history" where the change was > first introduced. > > In an email in this thread clime suggested a similar approach, only > instead of tags there would be a separate branch for each upstream > release. While that eliminates the need to allow force-pushes, for the > purposes of digging through the history it's the same thing. The only > difference is that instead of iterating over tags, I'd be iterating over > branches. > > The only other approach to source-git that I can think of is merging new > upstream releases instead of git-rebasing on top of them. That is the > approach that I originally thought would work, but one of Neal's > responses made me realize that this approach also has a significant > drawback - it makes distinguishing between downstream changes and > cherry-picked upstream changes hard. > > I was originally excited about source-git, however currently I don't see > an approach to source-git that would work for me and I don't think I'd > use it if it became available. And frankly, I think I wouldn't want > other people using it either because it would make understanding their > packages hard. > > I completely agree that dist-git is difficult to work with, but perhaps > instead of inventing something completely new, we could focus on making > working with dist-git easier by dropping the changlog and Release from > the specfiles and on creating tools for ourselves to make working with > patches easier? I'm currently looking into Quilt, mentioned by several > people here, to see if it could make my life easier at all. Just a > suggestion. > > Thanks everyone for this enlightening thread. > > Ondřej Lysoněk > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Is dist-git a good place for work?
On Thu, 14 May 2020 at 01:03, Ken Dreyer wrote: > > On Wed, May 13, 2020 at 4:29 PM clime wrote: > > Probably there are more variants but I see these three right now. I > > think variants 1 and 2 where the spec file is kept in dist-git but > > patches can be in source-git are more within our reach right now (but > > I might be wrong, variant 3 is also interesting). > > I think the best approach is to try your ideas on many different real > Fedora packages from many different upstreams, and refine the tools as > you go, documenting what works and what doesn't. Tools like tito and > rdopkg have the advantage of having been tested and hardened across > many different packages, Fedora releases, and RHEL versions. Yes, that's true but I think currently these tools assume the unpacked repo is outside of Fedora dist-git and import of the exported tarball into Fedora's git lookaside cache is needed. But I think we can optimize on this once the unpacked/source-git repos will be inside Fedora dist-git because we can use git submodules to link to them directly. Maybe at some point git submodules didn't have the best reputation? I remember something like this but in the present day, I think they are already heavily tried and something we could include into our workflow. So that's something worth looking at too :). But I think it's great that we have all these options like packit, rdopkg, tito, rpkg. > > In relation to rdopkg, I forgot to mention that Debian's > git-buildpackage tool uses a patch management model that is almost > identical to rdopkg for RPMs. Debian packagers create "patch-queue" > branches from the upstream project, and git-buildpackage can write out > a quilt-formatted series of .patch files into the debian packaging. It > is pretty fast to manage a large series of downstream patches this > way, rebase to new versions, etc. > > - Ken > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Is dist-git a good place for work?
On Wed, 13 May 2020 at 22:32, Ken Dreyer wrote: > > On Tue, May 12, 2020 at 6:20 PM clime wrote: > > When you do rdopkg new-version and you are asked to force push, is > > also the current master-patches HEAD tagged with the current package > > NVR? > > It's something that I have to do before I run "new-version". Here's > the command I ran today: > > $ rdopkg tag-patches --push > > And rdopkg performed the following commands for me: > > git tag python-jenkins-job-builder-3.2.0-1 master-patches > git push patches python-jenkins-job-builder-3.2.0-1 > > You can see the new tag here: > https://fedorapeople.org/cgit/ktdreyer/public_git/python-jenkins-job-builder.git/log/?h=master-patches > > rdopkg read the current NVR, tagged the tip of the master-patches > branch for me, and pushed that tag to the patches remote. > > > Somewhere I was expecting to see a lot of NVR tags for past sate of > > master-patches (i assume, you could have also f32-patches, > > f31-patches, epel8-patches, ... ?) but I don't see those tags :) that > > would form the mentioned "history of histories". > > You're correct, rdopkg supports branch names like "f32-patches", > "f31-patches", and "epel8-patches". In my case I only needed to patch > Rawhide, so I created "master-patches" there. > > You're right that I didn't create a ton of NVR tags. This package is a > super trivial example where I only started using this model to fix a > FTBFS, so I did not tag every NVR. The reason I did this was because > there are only a few patches and I did not expect to keep them in > Fedora very long, because I can easily rebase Rawhide to 3.3.0 soon, > and the upstream authors were going to ship 3.3.0 soon. When we ship > an unpatched upstream release, there's less utility to tagging the NVR > like that. > > For the ceph package in RH Ceph Storage, we've tagged over three > hundred NVRs with this system. We could probably go back and check > which old builds koji-gc has deleted, and then delete those Git tags > as well, if we want to clean up the ones that we never shipped. Ken, thank you very much for the detailed explanation. I understand now. I can see three variations on this approach for a per package setup 1) source-git repo, branches per upstream release + dist-git repo with the classic branching (f31, f32, ...) where the branch from source-git is included as a git submodule. You can have patches in source-git which are generic across all distribution release and you can also have a single distro-specific patches in dist-git as classic patch files, spec file would be in dist-git repo 2) the same thing but the source-git repo does not only have branches for upstream releases but also can have a branch -f32 when I need to patch something specifically for f32, again there is a dist-git repo with a spec file and the submodule for the respective branch (either generic or patched specifically for a certain distro release) 3) there is no dist-git repo with the classic branching structure and there is only source-git with a spec file included. This would require that we would need to define what e.g. f32 means somewhere else by pointing to specific refs in the source git from an f32 definition file. The best for start might be to actually still have some kind of dist-git repo with the classic branches but these would only point to some refs in source-git. I am not sure if there is something in git to enable to have a branch as a pointer to another branch in another repo but at worst we can use a specific file or a submodule again. Probably there are more variants but I see these three right now. I think variants 1 and 2 where the spec file is kept in dist-git but patches can be in source-git are more within our reach right now (but I might be wrong, variant 3 is also interesting). clime > > - Ken > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: CPE Weekly: 2020-05-11
On Tue, 12 May 2020 at 16:28, Clement Verna wrote: > > On Mon, 11 May 2020 at 12:55, Clement Verna wrote: >> >> >> >> On Mon, 11 May 2020 at 11:03, Fabio Valentini wrote: >>> >>> On Mon, May 11, 2020 at 10:42 AM Aoife Moloney wrote: >>> > ## GitForge Updates >>> > * We are tracking our progress here (nothing new added yet, fyi) >>> > https://fedoraproject.org/wiki/Git_forge_update >>> > * And the council are tracking the community issues in this ticket >>> > https://pagure.io/Fedora-Council/tickets/issue/292 >>> > * I have an Office hours IRC meeting slot on #fedora-meeting-1 @ >>> > 1400-1500 UTC every Thursday. Feel free to stop by and say hi! We can >>> > talk about Gitforge, or not :) >>> >>> I'm wondering, is actually anything happening here? >> >> >> Yes, I am still gathering the "pain points". I am going to open a ticket in >> the GitLab tracker (http://gitlab.com/gitlab-org/gitlab) this week, so that >> all the discussions about what we are asking will be public. > > > Hey so we now have a public issue[0] with GitLab that we are going to use to > drive the conversation. So if you are interested I highly encourage you to > follow that issue. Also this is currently focusing on the very basic features > of dist-git in purpose once we have a better idea on how these features looks > like in GitLab, we will be able to take a look at the rest of our needs. > > Hope that helps > > PS: If there is anything that you feel should be mentioned in the ticket, > please feel free to tell me about it so that I can look at editing the ticket. > > [0] - https://gitlab.com/gitlab-org/gitlab/-/issues/217350 Clement, thank you for your open approach. It's a welcomed change (I mean considering what we have seen from CPE leaders so far). > >> >> >>> >>> The "progress" being tracked in the wiki has been "nothing yet" since >>> the wiki page was announced a few weekly reports ago, and the linked >>> council ticket has not been updated in 2-3 weeks either :/ >>> >>> >>> Fabio >>> ___ >>> devel mailing list -- devel@lists.fedoraproject.org >>> To unsubscribe send an email to devel-le...@lists.fedoraproject.org >>> Fedora Code of Conduct: >>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ >>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines >>> List Archives: >>> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Is dist-git a good place for work?
M construction >using git format-patch or something comparing the Fedora dist-git branch >with the remote branch. Multiple remotes should be possible should new and >old versions of the upstream project need to be supported. Fedora dist-git >branches should know their remote. > > * I still want to be able to do 'fedpkg srpm' and get a standalone >ready-to-build SRPM file that I can carry around. > > * Possibly extend fedpkg to helper package maintainers submit patches from the >package to the upstream project. I very much like the ideas that you described although I have a slightly different view on implementation. - I would be very happy if the original spec files in dist-git described the building process. I.e. that I can open any spec file anywhere on src.fp.o and I will know what happens when that spec file gets built. I think this is a useful feature to quickly navigate ourselves and don't rely on context and implicit knowledge too much. As a concrete example, if something is going to dynamically generate patch files from git commits and also the respective "Patch:" lines, I think there should be something in the spec file telling me that this will happen. The advantage is explicitness and I could imagine that something could also have parameters that would e.g. enable to omit certain commit from patch generation, i.e. it has imho better flexibility when compared to some tool in our build pipeline that would just blindly paste Patch lines at a certain place in spec file. I think such behavior would be unexpected and unpleasant because the spec file itself doesn't give you the full picture of the build process and you need to have some extra knowledge that newcomers will not have. You can say: "Not a big deal" but these things add up. Also order of declarations sometimes plays a role in rpm spec file so pasting some lines somewhere might not always be a valid operation. - I think we should have an "importer" service that would download sources directly from upstream and placed them into dist-git's lookaside on demand, based on their checksum. It could also automatically check signatures when importing. When import is done, "sources" file would be updated. In future, it could also import git tags instead of just tarballs (i.e. it would basically just clone git repo and reset it to specific tag) and these would be the "source-git" repos (of course their more manual population by git push should be also possible). - I think we could try to have workflows that require no force pushes. They are imho not really good for cooperation among more than 1 people :) clime > > > PRs in dist-git would be more meaningful to me if we were able to have the > upstream repo as a remote in dist-git and our branches just an extension of > that. > > Thanks, > > -- > David Cantrell > Red Hat, Inc. | Boston, MA | EST5EDT > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Is dist-git a good place for work?
On Tue, 12 May 2020 at 23:06, Ken Dreyer wrote: > > On Tue, May 12, 2020 at 1:45 AM clime wrote: > > > > Ken, would it be, please, possible to provide links to the patch > > branches and mentioned dist-git repos. I would like to have a closer > > look. > > Sure. I can't share the links to the RH Ceph Storage dist-git repos, > so I will give one example where I used rdopkg in Fedora recently. > > Here is an example where I bumped the version of a Python package and > included some cherry-picked patches: > > https://src.fedoraproject.org/rpms/python-jenkins-job-builder/c/78b70d24cf65a4c7a100d3e56358ae22d3a6eaf6?branch=master > > At first glance, the two new patches I included there look like the > output from "git-format-patch", and that is because rdopkg wraps > git-format-patch for some operations. rdopkg automatically inserted > those into the .spec file, and it also formats them with some > compatibility options to preserve the .patch file formats between RHEL > 7's Git 1.8.3.1 + RHEL 8's Git 2.18.2 + Fedora's Git, so that it does > not matter what OS the packager is running. > > So that's the change in "master" (dist-git's rawhide branch), and > there is a corresponding "master-patches" branch to go along with > that: > > https://fedorapeople.org/cgit/ktdreyer/public_git/python-jenkins-job-builder.git/?h=master-patches > > In my dist-git clone on my laptop, I have three remotes, set up like this: > > $ git remote -v > origin > ssh://ktdre...@pkgs.fedoraproject.org/rpms/python-jenkins-job-builder > (fetch) > origin > ssh://ktdre...@pkgs.fedoraproject.org/rpms/python-jenkins-job-builder > (push) > patches > ssh://fedorapeople.org/home/fedora/ktdreyer/public_git/python-jenkins-job-builder.git > (fetch) > patches > ssh://fedorapeople.org/home/fedora/ktdreyer/public_git/python-jenkins-job-builder.git > (push) > upstreamhttps://opendev.org/jjb/jenkins-job-builder.git (fetch) > upstreamhttps://opendev.org/jjb/jenkins-job-builder.git (push) > > "rdopkg new-version" will update to the latest upstream version for > me. Specifically it looks at the upstream repo, finds the latest Git > tag, parses that tag string into a number, writes that number into the > .spec file, downloads and uploads the new upstream tarball, etc. It > will also rebase my "patches" branch for me and edit the Patch entries > as necessary. > > I haven't done that today for the sake of this example, but at some > point in the future I will run "rdopkg new-version", and it will pull > in 3.3.0 and eliminate those two patches, since they're both included > in version 3.3.0 upstream. > > In fact, you can try it on your computer if you set up the Git clones > like I've done above. If you run "rdopkg new-version", then rdopkg > will rewrite the "master-patches" branch, and then prompt you to > force-push this to the "patches" remote. You won't have SSH access to > push to my fedorapeople.org repo, so just imagine that is a team repo > where many people on my team can push :) This just a really simple > example with two patches in one small package. Thanks a lot! > Ceph we do this at a slightly different point of time. We use > "rdopkg tag-patches" to save each of the "patches" refs that we've > translated into patch series in dist-git. Each Git tag is the NVR of > the package. When you do rdopkg new-version and you are asked to force push, is also the current master-patches HEAD tagged with the current package NVR? Somewhere I was expecting to see a lot of NVR tags for past sate of master-patches (i assume, you could have also f32-patches, f31-patches, epel8-patches, ... ?) but I don't see those tags :) that would form the mentioned "history of histories". Thanks again clime > > - Ken > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Is dist-git a good place for work?
On Mon, 11 May 2020 at 18:59, Ken Dreyer wrote: > > On Sun, May 10, 2020 at 11:51 PM Petr Pisar wrote: > > How do you backport fixes? Do apply the fixes directly to dist-git? Or do > > you > > apply the fixes to a corresponding patches branch that you occur to have > > around till needed (e.g. till the hitorical code is supported) for the > > purpose > > of backporting? > > It's the latter. We use "git cherry-pick" to pick changes to our > "patches" branch, and then "rdopkg patch" writes those as .patch files > and PatchXXX entries in our .spec file in the corresponding dist-git > branch. Ken, would it be, please, possible to provide links to the patch branches and mentioned dist-git repos. I would like to have a closer look. Thanks clime > > At a general level, a typical Fedora packager performs three kinds of > operations in dist-git: > > 1) Rebasing to a new upstream version (eg. bumping the "Version" field > in httpd.spec from 2.4.43 to 2.4.44) > > 2) Fixing something in RPM packaging itself (eg. removing "Groups" > from httpd.spec file, fixing %check, etc) > > 3) Patching the source code (eg. cherry-picking a patch from > upstream). > > The current implementation of dist-git allows everyone and anyone to > very clearly audit all three of these actions. This kind of > transparency is really important to Fedora's goal of building a > trusted operating system. > > Upstream projects do ninja edits all the time. It's just too > convenient to force-push or move Git tags, etc. Sometimes upstream > authors have valid reasons for doing that kind of thing, but > downstream we have different incentives. The fact that we have strong > history preservation guarantees is one of the reasons I use Fedora. > > rdopkg has sub-commands to automate each of the three categories > above. For #3 (patching), in RH Ceph Storage we run the "rdopkg patch" > operations in Jenkins, because that is the most common operation by > far. > > I'm watching packit, and I am interested to try it out one day to > understand more about how it compares. I'm still not clear from this > thread what source-git is, or how it compares technically to what > we're doing with Ceph and OpenStack. > > - Ken > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Is dist-git a good place for work?
On Fri, 8 May 2020 at 20:05, Zbigniew Jędrzejewski-Szmek wrote: > > Hi, > I'm a bit late to the party, but here's my 2¢. > > On Mon, May 04, 2020 at 05:05:02PM +0200, Tomas Tomecek wrote: > > In the packit project, we work in source-git repositories. These are > > pretty much upstream repositories combined with Fedora downstream > > packaging files. > > I think source-git would be an interesting avenue to explore... > There's some hairy issues to figure out wrt. to rebasing of > downstream-only patches, but if that is solved, there would be great > potential to make certain styles of packaging much nicer. > > For more complicated projects, rebasing of patches would require some > git wizardry, but we'd reap the benefit of how good git is with > rebasing patches. From the workflows people described, it is clear > that many of us are doing some variant of a custom git branch to make > rebasing easier, building custom tooling around that. > > > Would there be an interest within the community, as opt-in, to have > > such source-git repositories created for respective dist-git > > repositories? The idea is that you would work in the source-git repo > > and then let packit handle synchronization with a respective dist-git > > repo. > > I agree with what Miro and others said about this: this brings a lot > of complication. I expect requirement to have synchronization both > ways is going to be a constant source of problems. We lose the > invariant that dist-git is the canonical source of truth. (Automatic > synchro is OK if it's just one way, but here it clearly needs to be > both ways because some maintainers would modify source-git and other > maintainers would modify dist-git.) > > IMO, source-git as a third repo in between the project and dist-git is > not useful. Instead, it would make sense when integrated with dist-git. I am curious. Zbyszek, what do you mean by "integrated with dist-git", here? > Tools like fedpkg and koji would need to learn to build a project > directly from this git repo, building a tarball on the fly. (What > smooge said about reliability: I wouldn't worry too much. 'git archive' > is reliable, and we'd be doing this locally, so this wouldn't be too > different from copying a tarball.) We then have a dist/source-git repo > that is very similar to upstream, and we don't have yet another component > in the system, but simple change how patches are represented in dist-git. > > (Hybrid approaches like Debian's quilt model don't make sense to me: > let's either use git or not use git, but doing both, and requiring > people to much with patch files is no better than our current dist-git.) > > > Our aim is to provide the contribution experience you have in > > GitHub when working on your packages. Dist-git would still be the > > authoritative source and a place where official builds are done - the > > source-git repo would work as a way to collaborate. We also don’t have > > plans right now to integrate packit into fedpkg. > > I don't get this. We need fewer (and better, more closely integrated) > tools, not yet another layer of helpers for other helpers. > > Zbyszek > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: sorting of git N-V-R tags in rpm package repositories
On Thu, 7 May 2020 at 20:50, Kevin Fenzi wrote: > > On Tue, May 05, 2020 at 06:46:48PM +0200, Florian Weimer wrote: > ...snip... > > > > The latest tag for a source package name wins for the Koji-generatged > > repository. I don't know what happens if different source packages > > build subpackages of the same name. > > koji operates on source packages, so as long as the source packages are > differently nameed, both subpackages would be there. What happens next > depends I think on what tool takes those packages and does things with > them. If they weere exactly the same, mergerepos_c would likely error > and the buildroot might be broken. Pungi might exit or might just pick > one. :) After some additional discussions git IRC, git mailing list, rpm-ecosystem IRC, and also taking into account how Koji works and what Florian was saying, I so far concluded that the sorting should be: 1) topological as a primary criterion 2) NVR-based where the "topological" criterion is not enough (multiple tags on the same commit, or when there are multiple "parallel" tags on merged-in branches) Doing just NVR rpm sort would work as well as long as people don't accidentally push some tag which would sort above even all future tags. Doing 1) as a primary criterion has an advantage that you can recover from this situation on your own even without deleting the remote tag (by pushing a new even empty commit and tagging it correctly this time :)). So doing 1+2 is more resilient against errors. libgit2-devel + rpm-devel were suggested to me so I'll try that to implement the tag sorter :). > > kevin > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Is dist-git a good place for work?
On Fri, 8 May 2020 at 16:22, Stephen John Smoogen wrote: > > On Fri, 8 May 2020 at 09:59, clime wrote: > > > > On Thu, 7 May 2020 at 20:58, Kevin Fenzi wrote: > > > > > > On Wed, May 06, 2020 at 08:39:19PM +0200, clime wrote: > > > ...snip... please folks... please trim your posts? :) > > > > > > > These are some great stats! > > > > > > > > But I would like to note that exploded repos (or source-git repos) > > > > have at least two other advantages. > > > > > > > > 1) they consume less space than tarballs for each version because > > > > objects in git repo are deduplicated > > > > > > But they consume tons more inodes which makes them painfull to > > > backup/restore/mirror. > > > > But maybe still less painful than to do this with upstream tarballs? > > No because the things that backups and rsync do works in a slow way. > We can do the backup the look-aside cache with tar-balls in a couple > of hours. We can also rsync that in the same amount of time. It takes > that long or longer to do that with a couple of git trees which are > much smaller in size but larger in file numbers. Every file in a git > tree is stat'd and while there is some deduplication, there is a lot > of files. Well from scratch, it will be much harder to rsync git repos but imho that changes once the initial rsync is done because then just new objects are transferred. + we could possibly use repospanner to mirror lookaside cache too? The deduplication in git is quite significant when compared to several tarballs from separate upstream releases. Anyway, I don't have the operative experience that you guys have so it's quite pointless for me to argue here. > > Could this be solved by moving to some other sort of file system > model... possibly but we > a) Have no time to pursue that investigation in a large enough size to > prove/disprove it > b) Have no money to purchase the equipment that these file systems work on. > > > > -- > Stephen J Smoogen. > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Is dist-git a good place for work?
On Thu, 7 May 2020 at 22:53, clime wrote: > > > > > In the rare occasion that I need to make downstream-only changes with > > patches, I usually just explode the upstream tarball, run "git init", > > then "git add .", "git commit -m import", apply my changes, and then > > do "git diff --patch > ../00-my-changes.patch" (if it's just one > > commit), or "git format-patch -o ../" if there are multiple commits, > > and then delete the exploded sources again. > > In any case, I think this functionality could be included in rpkg/fedpkg...? > > If there are no objections, I will open a ticket for this. > > Maybe also something from > https://pagure.io/glibc-maintainer-scripts/blob/master/f/glibc-sync-upstream.py > could be included too? I have opened https://pagure.io/rpkg/issue/502. But I think the source-git (or exploded sources) repos would be great to pursue as well. Out of the mentioned models to avoid history overwriting by force pushes (i.e. tags mentioned by Miro Hroncok and Hunor Csomortáni, branches mentioned by Simo Sorce, git merge -s ours mentioned by Florian Weimer), I like the branches approach the most with a slight tweak that branches are named according to upstream versions. I.e. for each "rebase", we create a new branch containing the new updated upstream sources and place our possibly updated patches on top. Effectively, a branch is just a single file in .git/refs/heads so it should be cheap and there will be no force pushes. Just the dist-git interface for working with branches should be ready for the fact that there may be a lot of them :). I think it is a small price for enabling this potentially very exciting, new approach. Best regards clime ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Is dist-git a good place for work?
On Thu, 7 May 2020 at 20:58, Kevin Fenzi wrote: > > On Wed, May 06, 2020 at 08:39:19PM +0200, clime wrote: > ...snip... please folks... please trim your posts? :) > > > These are some great stats! > > > > But I would like to note that exploded repos (or source-git repos) > > have at least two other advantages. > > > > 1) they consume less space than tarballs for each version because > > objects in git repo are deduplicated > > But they consume tons more inodes which makes them painfull to > backup/restore/mirror. But maybe still less painful than to do this with upstream tarballs? :) I guess it depends on average number of upstream releases per package. If the number is 1, then for sure tarballs will win. If the number is, let's say closer to 10, the storage size difference might be already quite significant to make the above operations harder. > > kevin > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Is dist-git a good place for work?
> In the rare occasion that I need to make downstream-only changes with > patches, I usually just explode the upstream tarball, run "git init", > then "git add .", "git commit -m import", apply my changes, and then > do "git diff --patch > ../00-my-changes.patch" (if it's just one > commit), or "git format-patch -o ../" if there are multiple commits, > and then delete the exploded sources again. In any case, I think this functionality could be included in rpkg/fedpkg...? If there are no objections, I will open a ticket for this. Maybe also something from https://pagure.io/glibc-maintainer-scripts/blob/master/f/glibc-sync-upstream.py could be included too? ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Is dist-git a good place for work?
Dne čt 7. kvě 2020 12:19 uživatel Vít Ondruch napsal: > > Dne 06. 05. 20 v 20:39 clime napsal(a): > > On Wed, 6 May 2020 at 13:21, Fabio Valentini > wrote: > >> On Wed, May 6, 2020 at 10:37 AM Vít Ondruch > wrote: > >>> > >>> Dne 05. 05. 20 v 18:37 Fabio Valentini napsal(a): > >>>> On Mon, May 4, 2020 at 5:06 PM Tomas Tomecek > wrote: > >>>> > >>>> Hi Tomas, > >>>> > >>>> I'll respond below with some of my experiences and opinions ... > >>>> > >>>>> Let’s talk about dist-git, as a place where we work. For us, > >>>>> packagers, it’s a well-known place. Yet for newcomers, it may take a > >>>>> while to learn all the details. Even though we operate with projects > >>>>> in a dist-git repository, the layout doesn’t resemble the respective > >>>>> upstream project. > >>>>> > >>>>> There is a multitude of tasks we tend to perform in a dist-git repo: > >>>>> * Bumping a release field for sake of a rebuild. > >>>>> * Updating to the latest upstream release. > >>>>> * Resolving CVEs. > >>>>> * Fixing bugs by… > >>>>> * Changing a spec file. > >>>>> * Pulling a commit from upstream. > >>>>> * Or even backporting a commit. > >>>>> * And more... > >>>>> > >>>>> For some tasks, the workflow is just fine and pretty straightforward. > >>>>> But for the other, it’s very gruesome - the moment you need to touch > >>>>> patch files, the horror comes in. The fact that we operate with patch > >>>>> files, in a git repository, is just mind-boggling to me. > >>>>> > >>>>> Luckily, we have tooling which supports the repository layout - > >>>>> `fedpkg prep`, `srpm` or `mockbuild` are such handy commands - you > can > >>>>> easily inspect the source tree or make sure your local change builds. > >>>>> > >>>>> Where am I getting with this? > >>>>> > >>>>> Over the years there have been multiple tools created to improve the > >>>>> development experience: > >>>>> rdopkg [r], rpkg-util [ru], tito [t] and probably much much more > (e.g. > >>>>> the way Fedora kernel developers work on kernel [k]). > >>>> (snip) > >>>> > >>>>> In the packit project, we work in source-git repositories. These are > >>>>> pretty much upstream repositories combined with Fedora downstream > >>>>> packaging files. An example: I recently added a project called > nyancat > >>>>> [n] to Fedora. I have worked [w] on packaging the project in the > >>>>> GitHub repo and then just pushed the changes to dist-git using packit > >>>>> tooling. These source-git repositories can live anywhere: we have > >>>>> support for GitHub right now and are working on supporting pagure. > >>>>> > >>>>> Would there be an interest within the community, as opt-in, to have > >>>>> such source-git repositories created for respective dist-git > >>>>> repositories? The idea is that you would work in the source-git repo > >>>>> and then let packit handle synchronization with a respective dist-git > >>>>> repo. Our aim is to provide the contribution experience you have in > >>>>> GitHub when working on your packages. Dist-git would still be the > >>>>> authoritative source and a place where official builds are done - the > >>>>> source-git repo would work as a way to collaborate. We also don’t > have > >>>>> plans right now to integrate packit into fedpkg. > >>>> So, in my experience, source-git might be a workable solution for > >>>> packages with *big* downstream modifications. However, for everything > >>>> else, I think it's a way to make it easy to accrue technical debt and > >>>> to do cargo-culting with downstream patches. > >>>> > >>>> The vast majority of packages has *no patches* (or at most, one or two > >>>> of them) > >> (snip) > >> > >>> I don't really want to argue with this point, I tend to agree. Just out > >>> of interest, do we have some statistics to support this? O:-) > >>
Re: Is dist-git a good place for work?
On Wed, 6 May 2020 at 21:00, Pierre-Yves Chibon wrote: > > On Wed, May 06, 2020 at 08:39:19PM +0200, clime wrote: > > But I would like to note that exploded repos (or source-git repos) > > have at least two other advantages. > > > > 1) they consume less space than tarballs for each version because > > objects in git repo are deduplicated > > 2) instead of downloading/uploading tarballs, you can just do > > something like: git pull --rebase upstream master; git push > > Just a note that this is not something you can do today since a rebase rewrite > history, so you would have to do `git push --force` which isn't allowed > currently. Good point. > So if we were to move forward with this model, we will need to find a solution > for the question that has led us to forbid force pushes until now. > > > Pierre > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Is dist-git a good place for work?
; 31: 6 > 32: 4 > 33: 3 > 34: 1 > 35: 4 > 37: 2 > 38: 1 > 41: 1 > 42: 2 > 46: 1 > 47: 1 > 48: 3 > 49: 1 > 50: 2 > 51: 1 > 53: 1 > 54: 1 > 66: 1 > 68: 1 > 71: 1 > 75: 1 > 78: 1 > 79: 1 > 85: 1 > 127: 1 > 170: 1 > > In relative terms: > > - 71% of packages have ZERO patches > - 15% have ONE patch > - 5% have TWO patches > - 3% have THREE patches > - 5% have MORE than THREE patches > > Most packages have none (71%) - or at most two - patches (91%, my > original "guess" for "vast majority"), some have 3-5 patches (5%), and > a minority (4%) has six patches or more. So it seems this backs up my > claim :) These are some great stats! But I would like to note that exploded repos (or source-git repos) have at least two other advantages. 1) they consume less space than tarballs for each version because objects in git repo are deduplicated 2) instead of downloading/uploading tarballs, you can just do something like: git pull --rebase upstream master; git push So they are imho more convenient to work with even if you don't have any patches. Continuing of communication with upstream should not be imposed by crappy experience when maintaining patches. It should be a question of work ethics and avoiding future conflicts with upstream. clime > > Fabio > > [0]: https://pkgs.fedoraproject.org/repo/rpm-specs-latest.tar.xz > > > > , and uses *unmodified upstream sources / tarballs*. > > > I never want to deal with exploded upstream sources, unless I'm > > > creating a patch for something. > > > > > > When it's an upstream commit that applies cleanly to the latest > > > sources, I'll just add it in the .spec file, and let the tooling > > > handle the rest. It's pretty neat to directly link to upstream commits > > > (it works with github and gitlab and pagure, as far as I know), and > > > let our tooling (spectool, fedpkg) do everything else. I don't have to > > > download, patch, or touch sources myself in any way for that. > > > > > > Unfortunately, in Ruby world, this unfortunately works less and less, > > because the released packages does not contain test suite these days. So > > if there is fix for some feature and associated test, then the patch has > > to be modified (the test part has to be stripped or split out). > > Otherwise I like this approach as well. > > > > > > > > > > When I need to make changes that I am able to push back upstream, I > > > don't do that in packaging, but fork upstream, do my changes, create a > > > pull request, and again point my .spec file to the patches from there. > > > No need to touch dist-git there, instead I'm working closely with > > > upstream. > > > > > > In the rare occasion that I need to make downstream-only changes with > > > patches, I usually just explode the upstream tarball, run "git init", > > > then "git add .", "git commit -m import", apply my changes, and then > > > do "git diff --patch > ../00-my-changes.patch" (if it's just one > > > commit), or "git format-patch -o ../" if there are multiple commits, > > > and then delete the exploded sources again. > > > > > > Having infrastructure for exploding sources from the package would be > > very interesting. > > > > Vít > > > > > > > > > > > > I maintain ~400 packages in fedora, and the only one with substantial > > > downstream modifications (about 10 patches on top of upstream) is > > > Jekyll (rubygem-jekyll), where I primarily disable tests for features > > > that are not enabled in fedora anyway (if I didn't want to run any > > > tests, I could just drop the number of patches to 1 or 2, making the > > > fork unnecessary - but I like running the tests). > > > > > > So while I agree that for *some* packages with *huge*, > > > non-upstreamable diffs between upstream and fedora the source-git > > > approach might work, I doubt that it would help in 99% of cases, or > > > even make it too easy for packagers to make more and more > > > downstream-only changes. > > > > > > Fabio > > > > > >> The main reason I am sending this is to gather feedback from all of > > >> you whether there is an interest in such a workflow. We don’t have > > >> concrete plans for Fedora right now but based on your feedback we > > >> could. > > >> > > >> > > >> [r] https://github.com/softwarefactory-project/rdopk
Re: sorting of git N-V-R tags in rpm package repositories
> Well, if you don't push the tag and you do a build, you will get N-V-R
> like foo-1.0-1.git.3.abcdef12. E.g. it won't be a clean N-V-R because
I meant "I.e." there, not "E.g."...just to be clear.
> it doesn't come from a tagged commit. If you push a tag and repeat a
> build from that same commit (or from that tag, see above), you will
> now get a clean N-V-R like foo-1.0-2. This is assuming Release: {{{
> git_dir_release }}} is used in the spec file to enable the automatic
> generation of release. The annotated tags represent releases. if you
> build an unreleased commit, you get a "work-in-progress" N-V-R and
> also changelog won't be populated with the latest changes. This
> workflow needs "pushing a tag" to be a build trigger so that it is
> convenient.
>
> Best regards
> clime
>
> >
> > Thanks,
> > Florian
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: sorting of git N-V-R tags in rpm package repositories
ush it later.
Well, if you don't push the tag and you do a build, you will get N-V-R
like foo-1.0-1.git.3.abcdef12. E.g. it won't be a clean N-V-R because
it doesn't come from a tagged commit. If you push a tag and repeat a
build from that same commit (or from that tag, see above), you will
now get a clean N-V-R like foo-1.0-2. This is assuming Release: {{{
git_dir_release }}} is used in the spec file to enable the automatic
generation of release. The annotated tags represent releases. if you
build an unreleased commit, you get a "work-in-progress" N-V-R and
also changelog won't be populated with the latest changes. This
workflow needs "pushing a tag" to be a build trigger so that it is
convenient.
Best regards
clime
>
> Thanks,
> Florian
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: sorting of git N-V-R tags in rpm package repositories
On Tue, 5 May 2020 at 18:46, Florian Weimer wrote: > > > Hey Florian, > > > > On Mon, 4 May 2020 at 10:03, Florian Weimer wrote: > >> > >> > as part of https://hackmd.io/kIje9yXTRdWITwP7cFK2pA (annotated tags > >> > pushed by package maintainers) effort, I revisited the sorting > >> > algorithm that is used to determine the "latest" tag for a given > >> > package which is needed to determine correct package version. > >> > Basically, if the current commit is tagged, then the N-V-R information > >> > from that tag name is directly used to render version or release > >> > (depending on macro usage). If the latest tag is on some older commit, > >> > we still use information from it but the version (or release) string > >> > will contain some appendices like .git.4.abcdef12 to mark a commit > >> > offset from that latest tag. Note that only tags accessible from the > >> > current branch tip when traversing git history backward are considered > >> > to pick the latest one (i.e. tags on other separate branches are not > >> > considered). > >> > >> Is this really necessary? Koji goes by latest tagged build, it does not > >> sort by NVR when constructing the buildroot. The same thing seems to > >> apply to composes (but I am less sure about that). > > > > > is it certain? I tried to poke around koji/kojira code, kojí IRC, and > > also logs at: https://koji.fedoraproject.org/koji/tasks which are very > > slow for owner:kojira, state:all but I couldn't really determine so > > far whether only the latest build of a package gets to "build" > > repository (that is used for builds subsequently) for a given tag. > > The latest tag for a source package name wins for the Koji-generatged > repository. I don't know what happens if different source packages > build subpackages of the same name. > > > If you build a package with an older version than what was there for > > previous build, does that older version overrides the newer one? I > > would need to poke more to find out. But it would be very interesting > > to know. > > Yes, if it gets tagged into the buildroot, it replaces a build with a > higher NVR. Good to know. But hard to say now how much inspiration can be really drawn from this. > > >> Tags can also be added retroactively and backdated. These things > >> conflict with the advantages you list (in particular, with NVR > >> auto-generation, git is not the sole source of truth). > > > > If the tag ordering function is done properly, I believe even > > retroactive tagging (i.e. tagging into past) and/or tag backdating > > would be supported and NVR auto-generation would work correctly. So I > > don't think it needs to conflict. But can you perhaps expand more on > > "Tags can also be added retroactively and backdated", please? > > I.e. why/when would one do that. > > No, you can push tags with incorrect dates. This can change the > auto-generation. It really depends on what the tag ordering function is. If the function does not consider dates at all, this wouldn't be a problem. > > You can only avoid this if you use data from commits (both current and > earlier) *on the same branch* exclusively for generating metadata (or > hash-linked from there). Everything else can get of sync and change > over time even if the commit hash stays the same, so the repository > state at a specific commit hash is longer the sole source of truth. > (Because you need to reconstruct that other state *at the right time*.) What do you mean by "everything else which can get out of sync and change"? If you are talking about tags (or refs in general), it's true that you can add tags into past which may or may not affect auto-generation depending on the ordering function. You could potentially also remove some tags if you haven't distribute them yet. But you should never push a tag with the same name that you have pushed before (https://git-scm.com/docs/git-tag#_on_re_tagging) - that makes the removal mentioned in the previous point kind of something which should perhaps be forbidden as well. So given this: 1) if we order tags by time of their creation, then if you create a new tag on the past commit, that tag will be considered the latest for the tagged commit itself and all the commits "above it" (i.e. commits further in git history) even if those commits have other tags (with older creation dates) assigned to them. These won't be used for auto-generation and instead the new intruder will be. If you try to build from some commit further in history now (e.g. by specifying an older tag for that commit which will lead to its checkout in the build system or just commit ID directly), the package N-V-Rs you start to get will look e.g. something like this: foo-1.0-3.git.8.abcdef12. foo-1.0-3 was the most recently created tag and we get the respective .git. suffix because the commit we want to build will be 8 commits past that latest tag and will have a hash starting with abcdef12. 2) if we order tags primarily by tagged-commit
Re: sorting of git N-V-R tags in rpm package repositories
On Tue, 5 May 2020 at 11:35, clime wrote:
>
> Hey Florian,
>
> On Mon, 4 May 2020 at 10:03, Florian Weimer wrote:
> >
> > > as part of https://hackmd.io/kIje9yXTRdWITwP7cFK2pA (annotated tags
> > > pushed by package maintainers) effort, I revisited the sorting
> > > algorithm that is used to determine the "latest" tag for a given
> > > package which is needed to determine correct package version.
> > > Basically, if the current commit is tagged, then the N-V-R information
> > > from that tag name is directly used to render version or release
> > > (depending on macro usage). If the latest tag is on some older commit,
> > > we still use information from it but the version (or release) string
> > > will contain some appendices like .git.4.abcdef12 to mark a commit
> > > offset from that latest tag. Note that only tags accessible from the
> > > current branch tip when traversing git history backward are considered
> > > to pick the latest one (i.e. tags on other separate branches are not
> > > considered).
> >
> > Is this really necessary? Koji goes by latest tagged build, it does not
> > sort by NVR when constructing the buildroot. The same thing seems to
> > apply to composes (but I am less sure about that).
>
> is it certain? I tried to poke around koji/kojira code, kojí IRC, and also
> logs at: https://koji.fedoraproject.org/koji/tasks which are very slow for
> owner:kojira, state:all but I couldn't really determine so far whether only
> the latest build of a package gets to "build" repository (that is used for
> builds subsequently) for a given tag.
>
> My (maybe wrong) understanding would be that all packages tagged into a
> certain tag (let's say f33) are included in the resulting "build" repository.
> Afterwards, dnf is used to create the actual buildroot for the package being
> built and dnf does (E-)N-V-R sort so it filters out any older package
> versions.
>
> If you build a package with an older version than what was there for previous
> build, does that older version overrides the newer one? I would need to poke
> more to find out. But it would be very interesting to know.
>
> >
> > So I don't see why you would need any sort of NVR sorting. Basically,
> > you can go back through the history and use the first tag you encounter
> > as the baseline.
>
> Tags also need to be sorted to render changelog records from their bodies in
> the correct order.
>
> What you suggest is the topological sort but it does not resolve the case
> when multiple tags are attached to the same commit. I would like to have the
> ordering defined for this scenario as well just in case.
>
> >
> > However, I think this whole approach is a bit fishy. Tags should be
> > pushed by Koji once a build completes, so that it is easy to identify
> > matching sources reliably, not by the packager.
>
> Well, it's true that with this approach you could make sure that the tags for
> the built packages are present after each build.
>
> That gives you a quick way to jump to a specific source code point based on a
> package name.
>
> With my suggested approach, pushing the tags would be an optional thing so it
> would only work for packages which do that.
>
> On the other hand, the build-induced tags are simply marks. They can't be
> used as a primary source for either changelog or release information. They do
> not need to be annotated, just simple tags. They can live a separate
> namespace which is what rpmautospec gentlemanly does at the moment.
>
> > Tags can also be added
> > retroactively and backdated. These things conflict with the advantages
> > you list (in particular, with NVR auto-generation, git is not the sole
> > source of truth).
>
> If the tag ordering function is done properly, I believe even retroactive
> tagging (i.e. tagging into past) and/or tag backdating would be supported and
> NVR auto-generation would work correctly. So I don't think it needs to
> conflict. But can you perhaps expand more on "Tags can also be added
> retroactively and backdated", please? I.e. why/when would one do that.
>
> Today, git is the sole source of truth for NVR information and changelog with
> the exception of %{dist}, which in most cases is pretty much equivalent to
> dist-git branch name. So, the annotated-tags proposal is such that it keeps
> git the source of truth for NVR and changelog but to avoid conflicts between
> branches and to allow for making those information dynamic, it moves that
> information from spec files out to git metadata.
>
> Thank you
> clime
>
Re: Is dist-git a good place for work?
On Tue, 5 May 2020 at 13:06, Miro Hrončok wrote: > > On 05. 05. 20 12:41, Tomas Tomecek wrote: > > Before I reply, I'd like to stress that we are still in a prototype > > phase - not everything is solved (clearly) and at this point, we > > experiment with the workflow mostly. > > Experimenting is cool. Go ahead. As long as this is totally opt-in and does > not > affect me as a contributor even if the main package maintainer chooses to use > it, all is good. > > > Luckily, force-pushes are not allowed in dist-git, which makes the > > update/sync process easier (knowing that history cannot be changed). > > Therefore when a new commit lands in dist-git, we'd just "transform" > > it to source-git and pushed it to the source-git repo. We could even > > ask all the contributors to rebase their PRs when this happens. > > On the other hand, when a new commit lands in the source-git repo, we > > could either transform and push to dist-git directly or open a PR. The > > maintainer should be in control of this process. > > That sounds overly complicated. I thought the idea is to make thing easier, > what > am I missing? > > > I understand the synchronisation adds friction to the overall > > architecture and may be the cause of many problems in the future - > > hence we are starting this discussion and using the technology > > ourselves to catch these issues asap. > > Good! > > > Miro, you are talking about conflicts: I'd say that conflicts on the > > git level are normal and git has solid tools to resolve them. For the > > use case of 2 different people changes the same thing, we would treat > > dist-git as the authoritative place and let the person in source-git > > know about the conflict. But this can happen nowadays easily as well: > > 2 different people can open the same PR or even push to dist-git > > directly while only one would succeed. > > No, actually, that is a misunderstanding. I was simply talking about > synchronization. But you have basically already answered my question above: > When > changes are done in dist-git, they will be somehow replicated in the > source-git > thing. > > > Petr, I should have probably stressed that our target is Fedora (or > > even all Red Hat operating systems). Yes, there are hundreds of > > distributions and we cannot solve their problems. We are open for > > collaboration though - we cannot drive changes in distributions which > > we don't know or use. > > Do I understand correctly that you no longer aim for the "put Fedora spec > files > upstream" thing, but rather, "create an intermediate upstream sources / fedora > scpec file" git repo hybrid? What's the benefit? > > > > See for example with Python. We have some patches (although we are trying to > get > rid of them) and rebasing those has proven to be quite tedious with patch > files > only. > > Hence, we keep this fork: https://github.com/fedora-python/cpython Imho, it would be nice if this could live on src.fp.o in a separate dedicated namespace for source repos. > > It has our patches on top. When new version is released, we rebase our branch > with git. We format-patch the commits and put them to dist git. > > If I had time, I'd create automation that does this for me. Unfortunately I > don't and I follow https://xkcd.com/1205/ > > In what way does keeping the spec file in our fork help us? > > -- > Miro Hrončok > -- > Phone: +420777974800 > IRC: mhroncok > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
sorting of git N-V-R tags in rpm package repositories
Hey Florian,
On Mon, 4 May 2020 at 10:03, Florian Weimer wrote:
>
> > as part of https://hackmd.io/kIje9yXTRdWITwP7cFK2pA (annotated tags
> > pushed by package maintainers) effort, I revisited the sorting
> > algorithm that is used to determine the "latest" tag for a given
> > package which is needed to determine correct package version.
> > Basically, if the current commit is tagged, then the N-V-R information
> > from that tag name is directly used to render version or release
> > (depending on macro usage). If the latest tag is on some older commit,
> > we still use information from it but the version (or release) string
> > will contain some appendices like .git.4.abcdef12 to mark a commit
> > offset from that latest tag. Note that only tags accessible from the
> > current branch tip when traversing git history backward are considered
> > to pick the latest one (i.e. tags on other separate branches are not
> > considered).
>
> Is this really necessary? Koji goes by latest tagged build, it does not
> sort by NVR when constructing the buildroot. The same thing seems to
> apply to composes (but I am less sure about that).
is it certain? I tried to poke around koji/kojira code, kojí IRC, and also
logs at: https://koji.fedoraproject.org/koji/tasks which are very slow for
owner:kojira, state:all but I couldn't really determine so far whether only
the latest build of a package gets to "build" repository (that is used for
builds subsequently) for a given tag.
My (maybe wrong) understanding would be that all packages tagged into a
certain tag (let's say f33) are included in the resulting "build"
repository. Afterwards, dnf is used to create the actual buildroot for the
package being built and dnf does (E-)N-V-R sort so it filters out any older
package versions.
If you build a package with an older version than what was there for
previous build, does that older version overrides the newer one? I would
need to poke more to find out. But it would be very interesting to know.
>
> So I don't see why you would need any sort of NVR sorting. Basically,
> you can go back through the history and use the first tag you encounter
> as the baseline.
Tags also need to be sorted to render changelog records from their bodies
in the correct order.
What you suggest is the topological sort but it does not resolve the case
when multiple tags are attached to the same commit. I would like to have
the ordering defined for this scenario as well just in case.
>
> However, I think this whole approach is a bit fishy. Tags should be
> pushed by Koji once a build completes, so that it is easy to identify
> matching sources reliably, not by the packager.
Well, it's true that with this approach you could make sure that the tags
for the built packages are present after each build.
That gives you a quick way to jump to a specific source code point based on
a package name.
With my suggested approach, pushing the tags would be an optional thing so
it would only work for packages which do that.
On the other hand, the build-induced tags are simply marks. They can't be
used as a primary source for either changelog or release information. They
do not need to be annotated, just simple tags. They can live a separate
namespace which is what rpmautospec gentlemanly does at the moment.
> Tags can also be added
> retroactively and backdated. These things conflict with the advantages
> you list (in particular, with NVR auto-generation, git is not the sole
> source of truth).
If the tag ordering function is done properly, I believe even retroactive
tagging (i.e. tagging into past) and/or tag backdating would be supported
and NVR auto-generation would work correctly. So I don't think it needs to
conflict. But can you perhaps expand more on "Tags can also be added
retroactively and backdated", please? I.e. why/when would one do that.
Today, git is the sole source of truth for NVR information and changelog
with the exception of %{dist}, which in most cases is pretty much
equivalent to dist-git branch name. So, the annotated-tags proposal is such
that it keeps git the source of truth for NVR and changelog but to avoid
conflicts between branches and to allow for making those information
dynamic, it moves that information from spec files out to git metadata.
Thank you
clime
>
> Thanks,
> Florian
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
CPE WEeekly: 2020-05-02
Dne po 4. kvě 2020 8:44 uživatel Clement Verna napsal: > > > On Sun, 3 May 2020 at 21:42, clime wrote: > >> On Sun, 3 May 2020 at 19:42, Aoife Moloney wrote: >> > >> > # CPE Weekly: 2020-05-02 >> > --- >> > title: CPE Weekly status email >> > tags: CPE Weekly, email >> > --- >> > >> > >> > >> > >> > Background: >> > The Community Platform Engineering group is the Red Hat team combining >> > IT and release engineering from Fedora and CentOS.Check out our teams >> > info here https://docs.fedoraproject.org/en-US/cpe/ >> > >> > >> > ## GitForge Updates >> > * We are tracking our progress here (nothing new added yet, fyi) >> > https://fedoraproject.org/wiki/Git_forge_update >> > * We are still doing a technical deep-dive with our own team on what >> > we need from GitLab and will have a technical plan developed and >> > publically available in the coming weeks - thanks again for your >> > patience, this will take some time to map out. >> >> Hello, >> >> what about using hackmd.io to track the progress of the plan in an >> open manner where people can contribute? >> > > The plan will be tracked in the Fedora wiki using the Change Request > template, everyone is more than welcome to contribute and provide comment > and I expect many iteration on that change proposal :-) > Good to hear. It would be even better if all options were still on the table. Then we could really say "Hurray" as a community. > >> >> I expected a restart of the git forge process because of the first one >> not being open and community-inclusive. >> >> Thank you >> clime >> >> > * Fedora have also released a blog post >> > https://communityblog.fedoraproject.org/fedora-council-and- >> the-git-forge/and >> > * And the council are tracking the community issues in this ticket >> > https://pagure.io/Fedora-Council/tickets/issue/292 >> > * We are looking at ways to engage closer with the community too so I >> > will have an *optional* office hours slot on #fedora-meeting @ >> > 1400-1500 UTC every Thursday. Feel free to stop by and say hi! We can >> > talk about Gitforge, or not :) >> > >> > >> > ## Releases!! >> > * F32 released! Congrats to all those who helped make this such an >> > awesome release :) >> > * Lenovo are releasing Fedora as a standard desktop offering! >> > * CentOS 7.8.2003 was released for x86_64, aarch64,ppc64, ppc64le and >> > armhfp architectures, including Cloud images (on >> > https://cloud.centos.org)! >> > >> > >> > >> > >> > >> > ### Data Centre Move >> > * Communishift is still out, est back online 11th May. >> > * Full amended schedule will be published week ending 8th May to >> > hackmd & will be sent to the devel & infra lists. >> > * Connectivity is now in place in IAD2 and should be in place in >> > RDU-CC over the weekend. >> > * In particular, a HUGE shout out to Stephen Smoogen who has been >> > working all the hours in every day for the last few weeks/months to >> > get this phase of the move operatoinal for the Fedora infrastructure - >> > we would not be able to do this without you Smooge :) >> > * This is literally a two man team of Kevin Fenzi and Stephen Smoogen, >> > who are carrying the weight of this infrastructure on their shoulders >> > and are invaluable to the success of this multi-team and multi-month >> > project, so thank you both. >> > * Given the pressures on the Infra folks, a general ask for patience >> > if your ticket / request / ping takes a little bit longer to reply to >> > >> > >> > >> > ### AAA Replacement >> > * The team will work with openSUSE to deploy FreeIPA + Noggin to >> > deploy it in their infra before we do! >> > * This is really exciting and the team are looking forward to seeing >> > how the solution works in another infrastructure! >> > * You can view the teams current, completed and backlog work here >> > https://github.com/orgs/fedora-infra/projects/6 >> > >> > >> > >> > ### Sustaining Team >> > * The team are using this dashboard to track their work >> > https://github.com/fedora-infra/mbbox/projects/1 >> > >> > * Mbbox Upgrade >> > * Zuul CI set up is done >> > * Koji-hub TLS support added to
sorting of git N-V-R tags in rpm package repositories
Hello, as part of https://hackmd.io/kIje9yXTRdWITwP7cFK2pA (annotated tags pushed by package maintainers) effort, I revisited the sorting algorithm that is used to determine the "latest" tag for a given package which is needed to determine correct package version. Basically, if the current commit is tagged, then the N-V-R information from that tag name is directly used to render version or release (depending on macro usage). If the latest tag is on some older commit, we still use information from it but the version (or release) string will contain some appendices like .git.4.abcdef12 to mark a commit offset from that latest tag. Note that only tags accessible from the current branch tip when traversing git history backward are considered to pick the latest one (i.e. tags on other separate branches are not considered). Originally, I sorted tags to find the "latest one" by using git directly and the criterion was -taggerdate which means the latest created tag by time goes first. I realized this will mess up the ordering in case a packager would create a tag into the past (i.e. on some older commit). That tag, while belonging to an older commit and probably having an older NVR, would be selected as the latest tag. I am not sure how much this is a practical problem but I don't think people would expect this so I started to look for some perhaps better sorting algorithm. One idea was to employ a topological sort, i.e. sort tags as they are encountered by traversing current branch tip backward in Git history. This would solve the above problem with tagging past commits but when there are multiple tags on the same commit another criterion and probably time-based one would be needed to determine order between those. In addition, pure topological sort of tags is kind of clumsy to do from the Git command-line. So I decided to to use git for-each-ref --sort='-taggerdate' --sort='-*committerdate' - i.e. use date of tagged commit as the primary criterion and creation of tag as the secondary criterion. This emulates topological sort given that people have the correct time set on their machines. Regrettably, there is a bug in git (which was very quickly fixed https://public-inbox.org/git/CAGqZTUvaiDQbiQ1dOoqLcy+GHZg+BuXY=Z+S=Dpsq=wm44d...@mail.gmail.com/T/#t) that causes -taggerdate not to be taken into account. I was able to work-around it by using git for-each-ref --format to print the relevant timestamps and then use sort command-line utility to sort the tags. The problem with this is that it has "only" 1-second resolution and therefore the order of tags created within one second is indeterminate. It's therefore not 100% predictable. And also it is time-based, so if someone has a wrong time set on his/her machine, it might not work. So finally, I started to think about sorting the tags by RPM sort. Their names have N-V-R format so it kind of makes sense to use rpm to sort them. You could tag into past and that tag will be considered the latest one only if it has the highest N-V-R. What do you think? What is the best sorting approach here? Thanks clime ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: CPE WEeekly: 2020-05-02
On Sun, 3 May 2020 at 19:42, Aoife Moloney wrote: > > # CPE Weekly: 2020-05-02 > --- > title: CPE Weekly status email > tags: CPE Weekly, email > --- > > > > > Background: > The Community Platform Engineering group is the Red Hat team combining > IT and release engineering from Fedora and CentOS.Check out our teams > info here https://docs.fedoraproject.org/en-US/cpe/ > > > ## GitForge Updates > * We are tracking our progress here (nothing new added yet, fyi) > https://fedoraproject.org/wiki/Git_forge_update > * We are still doing a technical deep-dive with our own team on what > we need from GitLab and will have a technical plan developed and > publically available in the coming weeks - thanks again for your > patience, this will take some time to map out. Hello, what about using hackmd.io to track the progress of the plan in an open manner where people can contribute? I expected a restart of the git forge process because of the first one not being open and community-inclusive. Thank you clime > * Fedora have also released a blog post > https://communityblog.fedoraproject.org/fedora-council-and-the-git-forge/and > * And the council are tracking the community issues in this ticket > https://pagure.io/Fedora-Council/tickets/issue/292 > * We are looking at ways to engage closer with the community too so I > will have an *optional* office hours slot on #fedora-meeting @ > 1400-1500 UTC every Thursday. Feel free to stop by and say hi! We can > talk about Gitforge, or not :) > > > ## Releases!! > * F32 released! Congrats to all those who helped make this such an > awesome release :) > * Lenovo are releasing Fedora as a standard desktop offering! > * CentOS 7.8.2003 was released for x86_64, aarch64,ppc64, ppc64le and > armhfp architectures, including Cloud images (on > https://cloud.centos.org)! > > > > > > ### Data Centre Move > * Communishift is still out, est back online 11th May. > * Full amended schedule will be published week ending 8th May to > hackmd & will be sent to the devel & infra lists. > * Connectivity is now in place in IAD2 and should be in place in > RDU-CC over the weekend. > * In particular, a HUGE shout out to Stephen Smoogen who has been > working all the hours in every day for the last few weeks/months to > get this phase of the move operatoinal for the Fedora infrastructure - > we would not be able to do this without you Smooge :) > * This is literally a two man team of Kevin Fenzi and Stephen Smoogen, > who are carrying the weight of this infrastructure on their shoulders > and are invaluable to the success of this multi-team and multi-month > project, so thank you both. > * Given the pressures on the Infra folks, a general ask for patience > if your ticket / request / ping takes a little bit longer to reply to > > > > ### AAA Replacement > * The team will work with openSUSE to deploy FreeIPA + Noggin to > deploy it in their infra before we do! > * This is really exciting and the team are looking forward to seeing > how the solution works in another infrastructure! > * You can view the teams current, completed and backlog work here > https://github.com/orgs/fedora-infra/projects/6 > > > > ### Sustaining Team > * The team are using this dashboard to track their work > https://github.com/fedora-infra/mbbox/projects/1 > > * Mbbox Upgrade > * Zuul CI set up is done > * Koji-hub TLS support added to CR > * Set up ReadTheDocs documentation - webhook missing for automatic build > * Identity container for testing > * Koji-builder CRD PR rebase - SSL authentication with koji-hub > * Refactor molecule test suite to share tests > > > > > > > > ## CentOS Updates > > ### CentOS CI > * OpenShift upgrade > * OpenStack to OpenNebula migration scripts > * Ansible playbooks to manage the creation and bootstrapping of > bare metal nodes with RHCOS > * Packaging work (fixing dependencies) > * Updated ci-user list on efforts we are putting for CI Infrastructure > > ### CentOS > * CentOS 7.8.2003 was released for x86_64, aarch64,ppc64, ppc64le and > armhfp architectures. Including Cloud images (on > https://cloud.centos.org) - > https://blog.centos.org/2020/04/release-centos-linux-7-2003/ > > > ### CentOS Stream > * Congratulations to Brian Stinson on his excellent session of Ask The > Expert, facilitated by Rich Bowen during Red Hat Summit - we hope you > caught it, it was really good! > * Using CentOS Stream in the CentOS QA group to prep for 8.2 > > > > > As always, feedback is welcome, and we will continue to look at ways > to improve the delivery and readability of this weekly report. > > >
Re: Proposal to enable spec file preprocessing step before srpm build
On Thu, 30 Apr 2020 at 20:40, Nicolas Mailhot via devel wrote: > > Le lundi 09 mars 2020 à 00:26 +0100, clime a écrit : > > Hi, > > > the code would fail because at that point, the > > git metadata is already missing (they are not included in srpms). > > Can’t storage of built-time information in srpms be fixed instead of > adding a whole new overlay over existing tools, that ties building to > Fedora infra? > > There are lots of practical applications of being able to record build > time info in srpms (for example, I’ve been asked to record the > buildroot state one way or another, and, IIRC, samba called rpm in its > scripts for the same reason). It took me a while to realize you don't mean BUILDTIME header info but state of build environment that affects build of srpms :). Anyway, maybe? I think this might be more handy for rpms rather than for srpms. In most cases, srpms are only source carriers? If this should be the way for dynamic changelog/release, you would still first need something that translates relevant git metadata into something which rpm can work with. I don't think rpm should do it on its own as it would become tied to git or any other future scm. Can you elaborate on what samba did? I would like to know because it sounded interesting. --- I don't think this proposal ties it to Fedora-Infra. Anyone using either mock or rpkg-based tool should be able to use it according to this proposal. > > Modules went the overlay route, it was not their best decision. I don't think it is a proof that overlay-based solution are wrong though. By the way, modularity, in my opinion, would be great if it produced rpms (created by mixing/overlaying rpms from built modulemd) instead of yum sub-repos. That way we wouldn't need to duplicate things like Obsoletes or Requires on another layer and it would get a proper binary format for distribution. > > Regards > > -- > Nicolas Mailhot > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: RFC: Feature macros (aka USE flags)
On Thu, 30 Apr 2020 at 10:40, Petr Šabata wrote:
>
> On Tue, Apr 28, 2020 at 3:40 AM clime wrote:
> >
> > Few more notes:
> >
> > I think an idea that build system could be simply passing
> > --with/--without options on mock's command-line should be
> > considered...then basically no change in spec files is needed (i.e. no
> > syntax change).
>
> Yes but I'd say such a change would be more complex and invasive. You
> would still need a place to store these values and make sure the
> systems (koji, mocks, rpmbuilds on the system, many more) would read
> the correct configs and pass the values.
Well, in the end, it would probably mean rpmbuild would have a new
option to consume build-config.yml with all the with/withouts, which
would be cool because the config could be serialized in its headers -
at least for those options that were actually used from the spec file.
A disadvantage is that bare/default rpmbuild would be using spec
defined defaults for the with options so you would need to add the new
--build-config option explicitly on command-line to actually build the
same rpms that are being built in mock or a build system (both could
do that by default). This seems quite alright to me, nevertheless. I
like that the method to influence a build is explicit and the config
can be imprinted into the built rpms. I think this will be otherwise a
problem if the .yml configs are translated into system-wide rpm
macros.
> I was trying to keep SPEC files as brief as possible. The first time
> you call any of these macros, the feature is initialized even without
> any preamble definitions. These are already compatible with
> --with/--without so you can override the defaults with them.
> Technically you could do just #%{use foo} in your preamble and then
> work with %with and %without, which is pretty much what you're
> suggesting. However, combining the two approaches feels more ugly and
> confusing to me than sticking to one.
I was rather suggesting using just a dedicated %with related preamble like:
%wIth_option_env to explicitly state: "hey, I am taking something from
the environment here."
It would give you a quick list in spec file what you can configure,
which is nice.
Anyway, it's quite hard to image %use and %with to co-exist. Ability
to affect %use by --with/--without options seems confusing so while I
like this initiative, it probably tries to change more than is really
required to change. Idk, I don't really have a strong opinion here. I
am glad you have built a cool prototype and showed it to the community
for review. I think that's an important thing here.
clime
>
>
> > > The yml files and translation from them into the actual macro files
> > > are nice but I would consider if the hw dependent default values can
> > > be added in future as a feature.
> > >
> > > The local_.yml file seems somewhat out of place to me. I think it
> > > could be rather kept as an idea for future and for now we could just
> > > start with only buildroot configs?
>
> It's a possibility for packages that want or need it. I wouldn't
> expect many if any local files in the beginning.
>
> P
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: [Test-Announce] Taskotron is EOL today
R.I.P Taskotron On Thu, 30 Apr 2020 at 10:01, Kamil Paral wrote: > > As previously announced [1], Taskotron [2] will be shut down today. See the > announcement and its discussion for more details and some background info. > > As a result, certain tests (beginning with “dist.“) will no longer appear for > new updates in Bodhi (in Automated Tests tab). Some of those tests (and even > new ones) will hopefully come back in the future with the help of Fedora CI > [3]. > > Thank you to everyone who contributed to Taskotron in the past or found our > test reports helpful. > > > [1] > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/TRQZCXI5IZH3GLJTCO56SMBX5UY6J6LL/ > [2] project page: https://pagure.io/taskotron > [3] https://docs.fedoraproject.org/en-US/ci/ > > ___ > test-announce mailing list -- test-annou...@lists.fedoraproject.org > To unsubscribe send an email to test-announce-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/test-annou...@lists.fedoraproject.org > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 32 is available now!
On Tue, 28 Apr 2020 at 16:08, Matthew Miller wrote: > > It’s here! We’re proud to announce the release of Fedora 32. > Thanks to the hard work of thousands of Fedora community > members and contributors, we’re celebrating yet another > on-time release! > > Read the official announcement at: > > * https://fedoramagazine.org/announcing-fedora-32/ Awesome! > > or just go ahead and grab it from: > > * https://getfedora.org/ > > -- > Matthew Miller > > Fedora Project Leader > ___ > announce mailing list -- annou...@lists.fedoraproject.org > To unsubscribe send an email to announce-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/annou...@lists.fedoraproject.org > ___ > announce mailing list -- annou...@lists.fedoraproject.org > To unsubscribe send an email to announce-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/annou...@lists.fedoraproject.org > ___ > devel-announce mailing list -- devel-annou...@lists.fedoraproject.org > To unsubscribe send an email to devel-announce-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel-annou...@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: RFC: Feature macros (aka USE flags)
Few more notes:
I think an idea that build system could be simply passing
--with/--without options on mock's command-line should be
considered...then basically no change in spec files is needed (i.e. no
syntax change).
It would be great if rpm remembered either the rpmbuild's command-line
in case the solution above is chosen or the set of with options (and
the final values for them) that were used during build if the
buildroot config approach is chosen. I think that would be very useful
for debugging and inspection of rpms.
...that's probably it from me.
clime
On Tue, 28 Apr 2020 at 03:06, clime wrote:
>
> On Mon, 27 Apr 2020 at 13:21, Petr Šabata wrote:
> >
> > Based on the recent discussions around %fedora/%rhel macros and ELN,
> > and %bcond generally being confusing to work with, I came up with a
> > distribution-wide feature that defines generic feature keywords and
> > associated helper macros that packages can check in build-time
> > conditionals.
> >
> > The key advantage here is the defaults are defined by the buildroot,
> > not the package. The package is just a building block.
> >
> > I'd like some input to improve this and unless this turns out to be a
> > really bad idea, I intend to submit it as a change proposal. Even
> > though the more packages use it the more beneficial it gets, it's, of
> > course, perfectly optional.
> >
> > Details in the gist:
> > https://gist.github.com/contyk/0f0585c57976ca18a293b3566408
>
> Ad. Dan Mach's presented spec file:
>
> > https://github.com/rpm-software-management/libdnf/blob/dnf-5-devel/libdnf.spec
>
> I really like how all the switches are defined almost on top of the
> spec file so I think lots of inspiration can be drawn from that.
>
> Now I also think that introduction of the new %use scheme for build
> options instead of the current with/without scheme might be an
> overkill.
>
> I think lines like these:
>
> %bcond_with html
> %bcond_without man
>
> could be replaced by:
>
> %with_option_env html 0
> %with_option_env man 1
>
> i.e. %with_option_env
>
> %with_option_env could be looking at an environmental buildroot
> setting with the possibility to be overridden on command-line by
> --with and --without as usual.
>
> There could be also:
> %with_option
>
> which wouldn't look at environment settings so it would be the same
> thing as %bcond_with/%bcond_without but less confusing.
>
> So with this, you wouldn't need to change conditions like:
>
> %if %{with foo}
> ...
> %endif
>
> and
>
> %if %{without foo}
> ...
> %endif
>
> ...so fewer changes and less work. I also think these conditions are
> quite fine (except relying on with_foo being either defined or
> undefined, which is quite funky but I don't think it is a reason to
> replace them).
>
> ---
>
> The yml files and translation from them into the actual macro files
> are nice but I would consider if the hw dependent default values can
> be added in future as a feature.
>
> The local_.yml file seems somewhat out of place to me. I think it
> could be rather kept as an idea for future and for now we could just
> start with only buildroot configs?
>
> Best regards!
> clime
>
> >
> > Cheers,
> > P
> > ___
> > devel mailing list -- devel@lists.fedoraproject.org
> > To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: RFC: Feature macros (aka USE flags)
On Mon, 27 Apr 2020 at 13:21, Petr Šabata wrote:
>
> Based on the recent discussions around %fedora/%rhel macros and ELN,
> and %bcond generally being confusing to work with, I came up with a
> distribution-wide feature that defines generic feature keywords and
> associated helper macros that packages can check in build-time
> conditionals.
>
> The key advantage here is the defaults are defined by the buildroot,
> not the package. The package is just a building block.
>
> I'd like some input to improve this and unless this turns out to be a
> really bad idea, I intend to submit it as a change proposal. Even
> though the more packages use it the more beneficial it gets, it's, of
> course, perfectly optional.
>
> Details in the gist:
> https://gist.github.com/contyk/0f0585c57976ca18a293b3566408
Ad. Dan Mach's presented spec file:
> https://github.com/rpm-software-management/libdnf/blob/dnf-5-devel/libdnf.spec
I really like how all the switches are defined almost on top of the
spec file so I think lots of inspiration can be drawn from that.
Now I also think that introduction of the new %use scheme for build
options instead of the current with/without scheme might be an
overkill.
I think lines like these:
%bcond_with html
%bcond_without man
could be replaced by:
%with_option_env html 0
%with_option_env man 1
i.e. %with_option_env
%with_option_env could be looking at an environmental buildroot
setting with the possibility to be overridden on command-line by
--with and --without as usual.
There could be also:
%with_option
which wouldn't look at environment settings so it would be the same
thing as %bcond_with/%bcond_without but less confusing.
So with this, you wouldn't need to change conditions like:
%if %{with foo}
...
%endif
and
%if %{without foo}
...
%endif
...so fewer changes and less work. I also think these conditions are
quite fine (except relying on with_foo being either defined or
undefined, which is quite funky but I don't think it is a reason to
replace them).
---
The yml files and translation from them into the actual macro files
are nice but I would consider if the hw dependent default values can
be added in future as a feature.
The local_.yml file seems somewhat out of place to me. I think it
could be rather kept as an idea for future and for now we could just
start with only buildroot configs?
Best regards!
clime
>
> Cheers,
> P
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: git_release rpkg macro - help needed
So I was thinking about it and the unification of code for git_release
and git_version makes sense according to my best knowledge.
Below is my reasoning. It would be great if somebody could check if it
is correct.
Note that in the end, the macros should be easy and intuitive to use.
The explanation below should only show that they are also robust
enough to cover all the possible use-cases with the assumption that
backward-compatible changes (i.e. adding more features/params) are OK
to make.
-
The only use-case which won't be covered by the provided lead/follow
functionality for the git_release (and git_version) macro is if one
would maintain two (or more) tag sets in a single branch.
E.g. in a single branch (called e.g. master), there would be tags:
prunerepo-1.14-1
prunerepo-1.14-2
prunerepo-1.14-master.1
prunerepo-1.14-master.2
One can work with just a single set when using git_release or
git_version macro. That's pretty much a physical limitation of each
macro. But one might want to be able to at least pick a set to work
with.
Now, this is currently a problem with the lead/follow scheme if the
set we want to pick is the one with the tags:
prunerepo-1.14-1
prunerepo-1.14-2
i.e. the "unprefixed release" variant.
We could try to use {{{ git_release lead="" }}} (i.e. just specify
lead should be empty, which is the default anyway) and hope this will
filter out the tags:
prunerepo-1.14-master.1
prunerepo-1.14-master.2
because follow is always the last component so everything before that
needs to be lead and we specified that lead should be empty, right?
Well, it doesn't currently work like that. Tag filtering that uses
lead will search for all tags, the release of which is literally
prefixed with ${lead}.
So if ${lead} is empty, all the tags from both sets will be matched so
we will recognize the last created one as our "latest tag" from which
release is derived. I.e. the resulting release string will be
alternating between values from {1, 2} and {master.1, master.2} based
on which tag (from which set) is currently the latest one.
While here it's a "bug" (or rather, I think, an unsupported use-case),
it's a feature in another context.
While the behavior doesn't allow for maintaining multiple tag sets
within a single branch (which should be rare imho) while picking the
desired one for version/release generation, it allows for direct
release/version derivation from the latest tag no matter what was the
lead in the latest tag.
I.e. if have this in my spec file:
Release: {{{ git_release }}}
(side note: in fact, the recommended form to use is git_dir_release)
...I can create whatever tag in the current branch, e.g.:
prunerepo-1.14-this_is_really_cool_package_pls_install.1
and when that spec file of mine is evaluated, I get:
Release: this_is_really_cool_package_pls_install.1
Another example: if the tag is:
prunerepo-1.14-42
I get:
Release: 42
I.e. no matter what the tag is, it will match and will be used for
generating the release field. In addition, if the last component of
the release is a number, auto-bumping will work as well.
I find this more pleasant and less constraining behavior. It allows
for setting up an initial tag with an arbitrary lead, which will be
then used for automatic bumping (given that follow is a number) and
the lead will be automatically carried over to the new tags.
If, however, at some point somebody needs the strict interpretation of
lead, an additional parameter of git_release/git_version can be
provided to enable it.
So the macros should be sufficiently robust after the behavior unification.
clime
clime
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedpkg: (scratch)-build forked repo directly in Koji
On Mon, 20 Apr 2020 at 14:24, Vít Ondruch wrote: > > > Dne 20. 04. 20 v 13:52 Ondrej Nosek napsal(a): > > > > On Mon, Apr 20, 2020 at 11:42 AM Vít Ondruch wrote: >> >> >> Dne 20. 04. 20 v 1:31 Ondrej Nosek napsal(a): >> >> Thanks for answers, comment in the text follows. >> >> On Tue, Apr 14, 2020 at 12:16 PM clime wrote: >>> >>> On Tue, 14 Apr 2020 at 12:05, Vít Ondruch wrote: >>> > >>> > >>> > Dne 14. 04. 20 v 0:13 Ondrej Nosek napsal(a): >>> > >>> > TLDR: Is $SUBJ function reasonable to implement in fedpkg? >>> > >>> > Hi, >>> > >>> > some time ago, fedpkg issue tracker got a request [1] for method, that >>> > allows direct builds. That means without sending srpms via "--srpm" >>> > argument. Currently, user's changes can be built: >>> > >>> > fedpkg scratch-build --srpm >>> > >>> > This way, fedpkg sends srpm file to koji. Without "--srpm", fedpkg sends >>> > just (git) hash id to koji. But fedpkg needs modification to send a right >>> > hash id for user changes. Additionally, koji doesn't allow building hash >>> > ids from forked repos. >>> > >>> > >>> > Even if this was possible, that would also mean that the sources have to >>> > be uploaded into look-a-side cache. Then it very much depend what is the >>> > content of the PR. If I am building Ruby nightly snapshots, I don't think >>> > it is fair to pollute look-a-side cache with them and I am afraid this is >>> > not the only case. I wish we had a way to override the look-a-side cache >>> > somehow >>> >>> If I understand correctly, this would have to be done, if sources >>> changed only, right? I.e. if there is a change just in patch or a spec >>> file, the sources could be fetched from the main project. >> >> >> Yes, just sources (and eventually other binary files) are uploaded to >> lookaside cache. In the case of specfile and patches, there is no lookaside >> modification. Fork shares the same lookaside cache with the main project. >> >>> >>> >>> Should there be a possibility to upload sources for a fork that would >>> then be moved to the main project when the pull request is merged? >> >> >> That sounds complicated to me and maybe it is not worth the intended result. >> Or I didn't find the right (easier) approach. ... New sources (and binaries) >> in fork need to be saved somewhere. >> - In a parallel lookaside (for forks) >> - In git repo (omitting lookaside) >> During the merge, some trigger would move the sources to the main lookaside. >> This creates a new file hash(es). And it would result in another commit done >> by a trigger. >> >> >> Why it should create new hash(es)? If the fork/PR contains an updated >> sources file (and it really should), then there is no reason for new commit. > > > Oh. In my last post, I wanted to say, that no commit is needed if you are > using main lookaside. Just in case you have parallel lookaside or other > storage, you have to upload sources from your alternative storage to main > lookaside and it would result in the extra commit. > But I looked at code how lookaside works internally. Hash is computed locally > and the upload path (except the host) should be the same for both lookasides. > So I realized that during merge, only download the sources from parallel > lookaside and upload it to the main lookaside would work. Without extra > commit. But is it a win? This might bring other problems. Timeouts when some > large source is processed? > Parallel lookaside is intended to not mess the main lookaside. So I think the > parallel lookaside should be cleaned somehow. > > > It should be cleaned as often as the stalled PRs? I know we don't have > infrastructure for this, but it would be logical answer to your question. Imho, you need to think also about repos from which pull requests are never created and that can have uploaded sources. > > > Vít > > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedpkg: (scratch)-build forked repo directly in Koji
On Mon, 20 Apr 2020 at 13:54, Ondrej Nosek wrote: > > > > On Mon, Apr 20, 2020 at 11:42 AM Vít Ondruch wrote: >> >> >> Dne 20. 04. 20 v 1:31 Ondrej Nosek napsal(a): >> >> Thanks for answers, comment in the text follows. >> >> On Tue, Apr 14, 2020 at 12:16 PM clime wrote: >>> >>> On Tue, 14 Apr 2020 at 12:05, Vít Ondruch wrote: >>> > >>> > >>> > Dne 14. 04. 20 v 0:13 Ondrej Nosek napsal(a): >>> > >>> > TLDR: Is $SUBJ function reasonable to implement in fedpkg? >>> > >>> > Hi, >>> > >>> > some time ago, fedpkg issue tracker got a request [1] for method, that >>> > allows direct builds. That means without sending srpms via "--srpm" >>> > argument. Currently, user's changes can be built: >>> > >>> > fedpkg scratch-build --srpm >>> > >>> > This way, fedpkg sends srpm file to koji. Without "--srpm", fedpkg sends >>> > just (git) hash id to koji. But fedpkg needs modification to send a right >>> > hash id for user changes. Additionally, koji doesn't allow building hash >>> > ids from forked repos. >>> > >>> > >>> > Even if this was possible, that would also mean that the sources have to >>> > be uploaded into look-a-side cache. Then it very much depend what is the >>> > content of the PR. If I am building Ruby nightly snapshots, I don't think >>> > it is fair to pollute look-a-side cache with them and I am afraid this is >>> > not the only case. I wish we had a way to override the look-a-side cache >>> > somehow >>> >>> If I understand correctly, this would have to be done, if sources >>> changed only, right? I.e. if there is a change just in patch or a spec >>> file, the sources could be fetched from the main project. >> >> >> Yes, just sources (and eventually other binary files) are uploaded to >> lookaside cache. In the case of specfile and patches, there is no lookaside >> modification. Fork shares the same lookaside cache with the main project. >> >>> >>> >>> Should there be a possibility to upload sources for a fork that would >>> then be moved to the main project when the pull request is merged? >> >> >> That sounds complicated to me and maybe it is not worth the intended result. >> Or I didn't find the right (easier) approach. ... New sources (and binaries) >> in fork need to be saved somewhere. >> - In a parallel lookaside (for forks) >> - In git repo (omitting lookaside) >> During the merge, some trigger would move the sources to the main lookaside. >> This creates a new file hash(es). And it would result in another commit done >> by a trigger. >> >> >> Why it should create new hash(es)? If the fork/PR contains an updated >> sources file (and it really should), then there is no reason for new commit. > > > Oh. In my last post, I wanted to say, that no commit is needed if you are > using main lookaside. Just in case you have parallel lookaside or other > storage, you have to upload sources from your alternative storage to main > lookaside and it would result in the extra commit. > But I looked at code how lookaside works internally. Hash is computed locally > and the upload path (except the host) should be the same for both lookasides. > So I realized that during merge, only download the sources from parallel > lookaside and upload it to the main lookaside would work. Without extra > commit. But is it a win? This might bring other problems. Timeouts when some > large source is processed? This should be ideally solved by hardlinks if the forks cache and the main cache can be on the same filesystem Otherwise copying is needed, which may be quite expensive. I.e. during merge you just create hardlink on the new location in the main repo that points to the source in the forks repo. > Parallel lookaside is intended to not mess the main lookaside. So I think the > parallel lookaside should be cleaned somehow. Periodically? Based on age? How > should I recognize if the file is safe to delete? Do users always use forked > projects as temporary so they wouldn't mind that unmerged source is deleted > from lookaside after one month? I think periodic cleaning is a very good idea here but people would need to know about it and be ok with it. >> >> Vít >> >> >> I think it is an unwanted situation. >> Some pollution of lookaside seems inevitable. But it happens even now >> without possibility t
Re: Fedpkg: (scratch)-build forked repo directly in Koji
On Mon, 20 Apr 2020 at 01:54, clime wrote: > > On Mon, 20 Apr 2020 at 01:33, Ondrej Nosek wrote: > > > > Thanks for answers, comment in the text follows. > > > > On Tue, Apr 14, 2020 at 12:16 PM clime wrote: > >> > >> On Tue, 14 Apr 2020 at 12:05, Vít Ondruch wrote: > >> > > >> > > >> > Dne 14. 04. 20 v 0:13 Ondrej Nosek napsal(a): > >> > > >> > TLDR: Is $SUBJ function reasonable to implement in fedpkg? > >> > > >> > Hi, > >> > > >> > some time ago, fedpkg issue tracker got a request [1] for method, that > >> > allows direct builds. That means without sending srpms via "--srpm" > >> > argument. Currently, user's changes can be built: > >> > > >> > fedpkg scratch-build --srpm > >> > > >> > This way, fedpkg sends srpm file to koji. Without "--srpm", fedpkg sends > >> > just (git) hash id to koji. But fedpkg needs modification to send a > >> > right hash id for user changes. Additionally, koji doesn't allow > >> > building hash ids from forked repos. > >> > > >> > > >> > Even if this was possible, that would also mean that the sources have to > >> > be uploaded into look-a-side cache. Then it very much depend what is the > >> > content of the PR. If I am building Ruby nightly snapshots, I don't > >> > think it is fair to pollute look-a-side cache with them and I am afraid > >> > this is not the only case. I wish we had a way to override the > >> > look-a-side cache somehow > >> > >> If I understand correctly, this would have to be done, if sources > >> changed only, right? I.e. if there is a change just in patch or a spec > >> file, the sources could be fetched from the main project. > > > > > > Yes, just sources (and eventually other binary files) are uploaded to > > lookaside cache. In the case of specfile and patches, there is no lookaside > > modification. Fork shares the same lookaside cache with the main project. > > Cool! > > > > >> > >> > >> Should there be a possibility to upload sources for a fork that would > >> then be moved to the main project when the pull request is merged? > > > > > > That sounds complicated to me and maybe it is not worth the intended > > result. Or I didn't find the right (easier) approach. ... New sources (and > > binaries) in fork need to be saved somewhere. > > - In a parallel lookaside (for forks) > > Yes, this is what I was thinking about. > > > - In git repo (omitting lookaside) > > During the merge, some trigger would move the sources to the main > > lookaside. This creates a new file hash(es). > > I don't think it creates new file hashes. I mean checksums of the > files should stay the same as the content stays the same, no? Maybe I > am omitting something. > > > And it would result in another commit done by a trigger. I think it is an > > unwanted situation. > > I think an additional commit should not be needed due to above. > > > Some pollution of lookaside seems inevitable. But it happens even now > > without possibility to take uploaded file back. > > Yes, that's true. > > --- > > I think the solution with fork-specific sources is mainly problematic. (somehow I managed to send the previous email prematurely) ...problematic (or might be problematic) because the forks can be created for other purposes than to create pull requests and then they can be also abandoned. So any sources that were uploaded for them would be there stuck probably forever. Maybe this is not that huge issue but something that should be considered. > > > > >> > >> > > >> > > >> > Vít > >> > > >> > > >> > The question to the community. Is reasonable to implement this way of > >> > building scratches? --srpm argument would still work as previously. > >> > > >> > There is a suggested PR for this here: [2]. It is not completed yet. > >> > > >> > Risks: > >> > * approach could confuse users. Users are used to work with fedpkg > >> > differently. > >> > * koji would have to allow these builds > >> > * more code complexity; currently there is a way how to reach your > >> > result even without this function > >> > > >> > Thanks > >> > Ondrej > >> > > >> > [1] https://pagure.io/fedpkg/
Re: Fedpkg: (scratch)-build forked repo directly in Koji
On Mon, 20 Apr 2020 at 01:33, Ondrej Nosek wrote: > > Thanks for answers, comment in the text follows. > > On Tue, Apr 14, 2020 at 12:16 PM clime wrote: >> >> On Tue, 14 Apr 2020 at 12:05, Vít Ondruch wrote: >> > >> > >> > Dne 14. 04. 20 v 0:13 Ondrej Nosek napsal(a): >> > >> > TLDR: Is $SUBJ function reasonable to implement in fedpkg? >> > >> > Hi, >> > >> > some time ago, fedpkg issue tracker got a request [1] for method, that >> > allows direct builds. That means without sending srpms via "--srpm" >> > argument. Currently, user's changes can be built: >> > >> > fedpkg scratch-build --srpm >> > >> > This way, fedpkg sends srpm file to koji. Without "--srpm", fedpkg sends >> > just (git) hash id to koji. But fedpkg needs modification to send a right >> > hash id for user changes. Additionally, koji doesn't allow building hash >> > ids from forked repos. >> > >> > >> > Even if this was possible, that would also mean that the sources have to >> > be uploaded into look-a-side cache. Then it very much depend what is the >> > content of the PR. If I am building Ruby nightly snapshots, I don't think >> > it is fair to pollute look-a-side cache with them and I am afraid this is >> > not the only case. I wish we had a way to override the look-a-side cache >> > somehow >> >> If I understand correctly, this would have to be done, if sources >> changed only, right? I.e. if there is a change just in patch or a spec >> file, the sources could be fetched from the main project. > > > Yes, just sources (and eventually other binary files) are uploaded to > lookaside cache. In the case of specfile and patches, there is no lookaside > modification. Fork shares the same lookaside cache with the main project. Cool! > >> >> >> Should there be a possibility to upload sources for a fork that would >> then be moved to the main project when the pull request is merged? > > > That sounds complicated to me and maybe it is not worth the intended result. > Or I didn't find the right (easier) approach. ... New sources (and binaries) > in fork need to be saved somewhere. > - In a parallel lookaside (for forks) Yes, this is what I was thinking about. > - In git repo (omitting lookaside) > During the merge, some trigger would move the sources to the main lookaside. > This creates a new file hash(es). I don't think it creates new file hashes. I mean checksums of the files should stay the same as the content stays the same, no? Maybe I am omitting something. > And it would result in another commit done by a trigger. I think it is an > unwanted situation. I think an additional commit should not be needed due to above. > Some pollution of lookaside seems inevitable. But it happens even now without > possibility to take uploaded file back. Yes, that's true. --- I think the solution with fork-specific sources is mainly problematic. > >> >> > >> > >> > Vít >> > >> > >> > The question to the community. Is reasonable to implement this way of >> > building scratches? --srpm argument would still work as previously. >> > >> > There is a suggested PR for this here: [2]. It is not completed yet. >> > >> > Risks: >> > * approach could confuse users. Users are used to work with fedpkg >> > differently. >> > * koji would have to allow these builds >> > * more code complexity; currently there is a way how to reach your result >> > even without this function >> > >> > Thanks >> > Ondrej >> > >> > [1] https://pagure.io/fedpkg/issue/282 >> > [2] https://pagure.io/fedpkg/pull-request/390 >> > >> > ___ >> > devel mailing list -- devel@lists.fedoraproject.org >> > To unsubscribe send an email to devel-le...@lists.fedoraproject.org >> > Fedora Code of Conduct: >> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ >> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines >> > List Archives: >> > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org >> > >> > ___ >> > devel mailing list -- devel@lists.fedoraproject.org >> > To unsubscribe send an email to devel-le...@lists.fedoraproject.org >> > Fedora Code of Conduct: >> > https://docs.fedoraproject.org/en-US/pro
