Re: What is the most time consuming task for you as packager?

[Sérgio Basto]

On Wed, 2020-12-30 at 14:41 -0800, Michel Alexandre Salim wrote:
> Howdy,
> 
> On Sat, 2020-12-26 at 20:39 +, Sérgio Basto wrote:
> > Hi, 
> > For me the most time consuming is monkey updates packages like kde
> > apps
> > , which every month or two we have a new release ( kde app 20.04.1
> > 20.04.2 20.08.0 , 20.12.00 etc )
> > 
> > I did some scripts to automate my builds , we got some software
> > like 
> > https://github.com/fedora-infra/the-new-hotness/ 
> > but the more I do, I always some variables that are different from
> > project to project , we need to know the version number, we may
> > need
> > to
> > download more than one source, we may need drop patches that we
> > know
> > that are already upstreamed, not all the package are build in same
> > branches so we need to know what branches we want update , we may
> > have
> > to add buildroot-overrides, we need add build to bodhi and fill
> > some
> > information , we need close bugs create made by hotness or other
> > users
> > etc
> > 
> > Examples of my scripts are usually in packages sources like 
> > 
> > https://src.fedoraproject.org/rpms/virtualbox-guest-additions/blob/master/f/update_vbox.sh
> >  
> > 
> > or (in attachment) scripts in very quick-and-dirty style 
> > 
> > So, combine tools like rpmdevtools , the-new-hotness , bodhi-client
> > etc
> > we improve building automation . 
> > 
> My use case is similar: I comaintain a list of packages that Facebook
> open-sources that need to be rebuilt in step (since sadly ABI
> stability
> is not currently promised).
> 
> I've cleaned-up my scripts:
> https://pagure.io/michel-slm/bulk-rebuild
> 
> and blogged about them:
> https://michel-slm.name/posts/2020-12-30-fedora-bulk-rebuild/
> 
> They mostly wrap around rpmdev-bumpspec, fedpkg, and bodhi right now.
> 
> Curious to see how much can be factored out and shared among
> different
> packagers that perform similar tasks. e.g. I see the GNOME packagers
> doing bulk updates too.
> 
> One observation (also in the post):
> 
> The CLIs for fedpkg and koji is currently meant for human, not
> machine,
> consumptions. Invoking them from Bash scripts and trying to use the
> output (e.g. getting the name of that side tag) involves some brittle
> parsing. I plan to rewrite this in Python anyway, but it might be
> useful to add an output formatter to some commands where it makes
> sense
> (e.g. request-side-tag or chain-build).

I was looking at your scripts and I saw a reference to bodhi-cli,
yesterday I found out that fedpkg also has this option. 

fedpkg update --help

usage: fedpkg update [-h] [--type
{bugfix,security,enhancement,newpackage}] [--request {testing,stable}]
[--bugs BUGS [BUGS ...]] [--notes NOTES]
 [--disable-autokarma] [--stable-karma KARMA] [
--unstable-karma KARMA] [--not-close-bugs] [--suggest-reboot] [--no-
require-bugs]
 [--no-require-testcases]

This will create a bodhi update request for the current package n-v-r.

There are two ways to specify update details. Without any argument from
command
line, either update type or notes is omitted, a template editor will be
shown
and let you edit the detail information interactively.

Alternatively, you could specify argument from command line to create
an update
directly, for example:

fedpkg update --type bugfix --notes 'Rebuilt' --bugs 1000 1002

When all lines in template editor are commented out or deleted, the
creation
process is aborted. If the template keeps unchanged, fedpkg continues
on creating
update. That gives user a chance to confirm the auto-generated notes
from
change log if option --notes is omitted.

optional arguments:
  -h, --helpshow this help message and exit
  --type {bugfix,security,enhancement,newpackage}
Update type. Template editor will be shown if
type is omitted.
  --request {testing,stable}
Requested repository.
  --bugs BUGS [BUGS ...]
Bug numbers. If omitted, bug numbers will be
extracted from change logs.
  --notes NOTES Update description. Multiple lines of notes
could be specified. If omitted, template editor will be shown.
  --disable-autokarma   Karma automatism is enabled by default. Use
this option to disable that.
  --stable-karma KARMA  Stable karma. Default is 3.
  --unstable-karma KARMA
Unstable karma. Default is -3.
  --not-close-bugs  By default, update will be created by enabling
to close bugs automatically. If this is what you do not want, use this
option to disable
the default behavior.
  --suggest-reboot  Suggest user to reboot after update. Default is
False.
  --no-require-bugs Disables the requirement that all of the bugs
in your update have been confirmed by testers. Default is True.
  --no-require-testcases
Disables the requirement that this update
passes all test cases before reaching stable. Default is True.

Re: What is the most time consuming task for you as packager?

[Nikola Forró]

On Sat, 2020-12-26 at 20:39 +, Sérgio Basto wrote:
> Hi, 
> For me the most time consuming is monkey updates packages like kde
> apps, which every month or two we have a new release ( kde app
> 20.04.1 20.04.2 20.08.0 , 20.12.00 etc )
> 
> I did some scripts to automate my builds , we got some software like 
> https://github.com/fedora-infra/the-new-hotness/ 
> but the more I do, I always some variables that are different from
> project to project , we need to know the version number, we may need
> to download more than one source, we may need drop patches that we
> know that are already upstreamed, not all the package are build in
> same branches so we need to know what branches we want update , we
> may have to add buildroot-overrides, we need add build to bodhi and
> fill some information , we need close bugs create made by hotness or
> other users etc
> 
> Examples of my scripts are usually in packages sources like 
> 
> https://src.fedoraproject.org/rpms/virtualbox-guest-additions/blob/master/f/update_vbox.sh
>  
> 
> or (in attachment) scripts in very quick-and-dirty style 

Have you tried rebase-helper? [1] It won't solve everything, but you could find 
it useful.
However particularly in case of virtualbox-guest-additions there is an
issue with the extra "a" in source name. I'm open to suggestions on how
to deal with that :)

Regards,
Nikola

[1] https://github.com/rebase-helper/rebase-helper


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Fabio Valentini]

On Wed, Dec 30, 2020 at 11:42 PM Michel Alexandre Salim
 wrote:
>
> Howdy,
> One observation (also in the post):
>
> The CLIs for fedpkg and koji is currently meant for human, not machine,
> consumptions. Invoking them from Bash scripts and trying to use the
> output (e.g. getting the name of that side tag) involves some brittle
> parsing. I plan to rewrite this in Python anyway, but it might be
> useful to add an output formatter to some commands where it makes sense
> (e.g. request-side-tag or chain-build).

+1000

It would be really really nice if some CLI tools (koji, fedpkg, etc.)
had a switch to print machine-readable output instead of
human-readable output. I'm also resorting to parsing human-readable
stdout from some of those tools in my scripts ... Probably that
machine-readable format would turn out to be JSON of some kind? It
seems to be the standard for this interoperability stuff now, and
there's good support for bash (jq), python (json from stdlib), and all
kinds of other languages.

Fabio
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Michel Alexandre Salim]

On Mon, 2020-12-21 at 13:02 +0100, Miroslav Suchý wrote:
> Dne 19. 12. 20 v 21:24 Luya Tshimbalanga napsal(a):
> > Just an idea of a form application that generates the spec file for
> > newcomers as an example.
> 
> Something like
>   https://xsuchy.github.io/rpm-spec-wizard/
> ?
> 
rpmdev-newspec also works fine. The problem is that it's meant to be
cross-platform and so the specs might not always align with the latest
Fedora Packaging Guidelines.

Sometimes it's lagging behind too; perhaps we could make it part of the
process for updating packaging guidelines to also verify that the
templates are up to date?

Also, templates only exist for some usecases; from the help:

  -t TYPE, --type TYPE
 Force use of the TYPE spec template. The default is
guessed
 from , falling back to "minimal" if the
guesswork
 does not result in a more specific one or if 
is not
 given. Available types:
 dummy lib minimal ocaml perl php-pear python R
ruby

Best regards,

-- 
Michel Alexandre Salim
profile: https://keyoxide.org/mic...@michel-slm.name
chat via email: https://delta.chat/
GPG key: 5DCE 2E7E 9C3B 1CFF D335 C1D7 8B22 9D2F 7CCC 04F2


signature.asc
Description: This is a digitally signed message part
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Michel Alexandre Salim]

Howdy,

On Sat, 2020-12-26 at 20:39 +, Sérgio Basto wrote:
> Hi, 
> For me the most time consuming is monkey updates packages like kde
> apps
> , which every month or two we have a new release ( kde app 20.04.1
> 20.04.2 20.08.0 , 20.12.00 etc )
> 
> I did some scripts to automate my builds , we got some software like 
> https://github.com/fedora-infra/the-new-hotness/ 
> but the more I do, I always some variables that are different from
> project to project , we need to know the version number, we may need
> to
> download more than one source, we may need drop patches that we know
> that are already upstreamed, not all the package are build in same
> branches so we need to know what branches we want update , we may
> have
> to add buildroot-overrides, we need add build to bodhi and fill some
> information , we need close bugs create made by hotness or other
> users
> etc
> 
> Examples of my scripts are usually in packages sources like 
> 
> https://src.fedoraproject.org/rpms/virtualbox-guest-additions/blob/master/f/update_vbox.sh
>  
> 
> or (in attachment) scripts in very quick-and-dirty style 
> 
> So, combine tools like rpmdevtools , the-new-hotness , bodhi-client
> etc
> we improve building automation . 
> 
My use case is similar: I comaintain a list of packages that Facebook
open-sources that need to be rebuilt in step (since sadly ABI stability
is not currently promised).

I've cleaned-up my scripts:
https://pagure.io/michel-slm/bulk-rebuild

and blogged about them:
https://michel-slm.name/posts/2020-12-30-fedora-bulk-rebuild/

They mostly wrap around rpmdev-bumpspec, fedpkg, and bodhi right now.

Curious to see how much can be factored out and shared among different
packagers that perform similar tasks. e.g. I see the GNOME packagers
doing bulk updates too.

One observation (also in the post):

The CLIs for fedpkg and koji is currently meant for human, not machine,
consumptions. Invoking them from Bash scripts and trying to use the
output (e.g. getting the name of that side tag) involves some brittle
parsing. I plan to rewrite this in Python anyway, but it might be
useful to add an output formatter to some commands where it makes sense
(e.g. request-side-tag or chain-build).

Thanks,

-- 
Michel Alexandre Salim
profile: https://keyoxide.org/mic...@michel-slm.name
chat via email: https://delta.chat/
GPG key: 5DCE 2E7E 9C3B 1CFF D335 C1D7 8B22 9D2F 7CCC 04F2


signature.asc
Description: This is a digitally signed message part
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Dan Čermák]

Miroslav Suchý  writes:

> I am looking for challenges for upcoming year - what I and my team should 
> enhance. I have some ideas, but I want to hear
> yours.
>
> What you - as Fedora packager - find most time consuming on packaging?
> Where you will welcome more simplicity or automation?

Besides what was already said, I'd welcome the option to start builds &
updates directly from pull requests on pagure. If someone sends me a
pull request that only needs a fedpkg build && fedpkg update, then it
would be terrific if I could trigger this from pagure's web UI directly
and be done with it.


Cheers,

Dan


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Dan Čermák]

Michael Cronenworth  writes:

> On 12/15/20 4:29 PM, Miroslav Suchý wrote:
>> What you - as Fedora packager - find most time consuming on packaging?
>> Where you will welcome more simplicity or automation?
>
> Pushing updates requires too many steps, IMHO.
>
> 1. 
> 2. fedpkg mockbuild
> 3. 
> 4. fedpkg commit -p
> 5. fedpkg build
> 6. fedpkg update ...

100% agree with this and add one to two fedpkg switch-branch commands to
that with a bunch of git merge and git cherry-pick.


> Can we make the Bodhi update notes part of the git repo? Ex. ChangeLog.md
> If not, could a file be referenced instead of a "--notes" argument? Ex. 
> --notes-file 
> ChangeLog.md
> Stretch goal: Bodhi update defaults (inc. notes?) as part of git,
> too. Ex. update.yml

Afaik there was the good idea to use annotated git tags for that. Thus
this whole thing would be condensed to:
git commit -m $commit_msg
git tag -a -m $update_msg
fedpkg push
(-> build is run and a bodhi update is triggered)


Cheers,

Dan


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Luya Tshimbalanga]

On 2020-12-21 4:02 a.m., Miroslav Suchý wrote:

Dne 19. 12. 20 v 21:24 Luya Tshimbalanga napsal(a):

Just an idea of a form application that generates the spec file for newcomers 
as an example.

Something like
   https://xsuchy.github.io/rpm-spec-wizard/
?

I do not propagate it too much as the first page needs a lot of love - it seems 
empty, but there is button in upper left
corner, which does the magic.
Cool!! That is a much needed tool for new packagers!! The first page 
could start with a title "RPM SPEC generator", a brief description of 
its purpose and then the button at the bottom.


--
Luya Tshimbalanga
Fedora Design Team
Fedora Design Suite maintainer
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Sérgio Basto]

Hi, 
For me the most time consuming is monkey updates packages like kde apps
, which every month or two we have a new release ( kde app 20.04.1
20.04.2 20.08.0 , 20.12.00 etc )

I did some scripts to automate my builds , we got some software like 
https://github.com/fedora-infra/the-new-hotness/ 
but the more I do, I always some variables that are different from
project to project , we need to know the version number, we may need to
download more than one source, we may need drop patches that we know
that are already upstreamed, not all the package are build in same
branches so we need to know what branches we want update , we may have
to add buildroot-overrides, we need add build to bodhi and fill some
information , we need close bugs create made by hotness or other users
etc

Examples of my scripts are usually in packages sources like 

https://src.fedoraproject.org/rpms/virtualbox-guest-additions/blob/master/f/update_vbox.sh
 

or (in attachment) scripts in very quick-and-dirty style 

So, combine tools like rpmdevtools , the-new-hotness , bodhi-client etc
we improve building automation . 

Thank you. 

 
-- 
Sérgio M. B.


update_from_bugzilla.sh
Description: application/shellscript


update_package.sh
Description: application/shellscript
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Kevin Fenzi]

On Sat, Dec 26, 2020 at 01:04:50AM +0100, Miro Hrončok wrote:
> On 23. 12. 20 0:11, Kevin Fenzi wrote:
> > Cool. The help didn't get me this at all.;)
> > 
> >--from-git FROM_GIT   regexes for packages to build from git
> > 
> > Perhaps that should be "copr-name packager-name:packagename:branchname"
> > ?
> 
> As a side note, the --from-git option is not what the "copr-name
> packager-name:packagename:branchname" is. The --from-git option is an extra
> option you specify to change the way dependent packages are built. The
> "copr-name packager-name:packagename:branchname" are the positional
> arguments of the script:
> 
> positional arguments:
>   copr  copr repo to use (and create)
>   project   projects to build (in order); format: USER:REPO:BRANCH

Ah ha. I completely misread it then. Thanks. 

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Miro Hrončok]

On 23. 12. 20 0:11, Kevin Fenzi wrote:
Cool. The help didn't get me this at all.;)  


   --from-git FROM_GIT   regexes for packages to build from git

Perhaps that should be "copr-name packager-name:packagename:branchname"
?


As a side note, the --from-git option is not what the "copr-name 
packager-name:packagename:branchname" is. The --from-git option is an extra 
option you specify to change the way dependent packages are built. The 
"copr-name packager-name:packagename:branchname" are the positional arguments of 
the script:


positional arguments:
  copr  copr repo to use (and create)
  project   projects to build (in order); format: USER:REPO:BRANCH


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Mikolaj Izdebski]

On Thu, Dec 17, 2020 at 9:11 PM Kevin Fenzi  wrote:
>
> On Wed, Dec 16, 2020 at 03:56:25PM -0500, Neal Gompa wrote:
> ...snip...
> > > > What I'd really like would be a "test mass rebuild" process, where a
> > > > prospective package is uploaded and everything that depends on it is
> > > > automatically rebuilt (ideally after creating the dependency graph so
> > > > each individual package doesn't get rebuilt until its dependencies are
> > > > finished building).
> > > >
> > > > Creating the dependency graph by hand is fairly tedious, but maybe I'm
> > > > missing an automated way. The point of creating that graph is to avoid
> > > > wasting time and power doing and redoing builds that will fail until
> > > > something else has been built (which is the problem with mock's
> > > > --chain command, if I understand correctly).
> > > >
> > > > Once I have that graph, I use Make to control the process, because it
> > > > handles the dependency graph, as well as parallelism, and not
> > > > rebuilding things unnecessarily.
> > >
> > > Yeah, all this ^
> > >
> >
> > So I've written tools for doing this, and Igor has written tools for
> > doing this, but it seems like people think that this is "impossible"
> > and so the effort goes nowhere despite several PoCs.
> >
> > If we're interested in this again for real this time, I could try to
> > dig out my old code for it, but we might be better off just pulling
> > out Koschei's code and turning it into something that Koji's
> > chain-build command and Mock's --chain option use to sort through
> > package sets and build them correctly.
>
> Can you expand on which 'this' you mean? Getting the build
> order/dependency graph? Or a tool to use that to rebuild everything and
> tell you what failed? or ?
>
> I don't think you can ever be 100% on dependency graph/build order,
> because there's sometimes bootstraps or loops in there. :(
>
> Anyhow I would love to be able to locally build a new library and run
> 'fedpkg does-it-blend foo.src.rpm' and have it tell me exaclty what
> other packages that breaks.

That feature was implemented [1] in Koschei long time ago, but it is
not enabled in Fedora production instance of Koschei due to Copr not
completing RFR [2]. It should be deployed in Fedora staging instance
of Koschei. You can build one or more package updates locally or in
Copr and send request to Koschei to test the set of packages. Koschei
will check which packages are affected by the update by resolving
dependencies and running builds in Copr. Koschei provides an overview
page saying which packages were affected (broken or fixed) by the
update.

Another variant of this feature was implemented [3] more recently
without reliance on Copr - it makes use of Koji side tags. It was
intended to be integrated with CI and gating - Koschei can test side
tags and emit results to messaging bus in CI format [4] that is
understood by CI system and can be used by packagers to implement
gating for their packages. Again, this feature is not enabled in
production, only in staging.

[1] https://github.com/fedora-infra/koschei/issues/113
[2] https://pagure.io/fedora-infrastructure/issue/5166
[3] https://pagure.io/fedora-ci/general/issue/45#comment-606344
[4] https://pagure.io/fedora-ci/messages

>
> kevin
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Kevin Fenzi]

On Mon, Dec 21, 2020 at 11:03:36AM +0100, Vít Ondruch wrote:
> 
> Dne 17. 12. 20 v 21:17 Kevin Fenzi napsal(a):
> > 
> > After 1 day the rpms are removed.
> > After 7 days the logs are removed.
> > 
> > Non koschei scratch builds (both logs and rpms) are kept for 14 days.
> > 
> > Would keeping the logs for 14 days help?
> 
> 
> That would be definitely appreciated.

Done.

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Kevin Fenzi]

On Tue, Dec 22, 2020 at 11:15:53AM +0100, Miro Hrončok wrote:
> On 21. 12. 20 19:36, Kevin Fenzi wrote:
> > On Thu, Dec 17, 2020 at 09:21:15PM +0100, Fabio Valentini wrote:
> > > As Miro mentioned, I've also developed scripts to handle this "does
> > > this update break anything" for the Stewardship / Java SIG, because -
> > > at least at first - we didn't have big enough egos / enough confidence
> > > to just push updates to rawhide without testing excessively them
> > > first:
> > > https://github.com/fedora-stewardship/fedora-stewardship.github.io/blob/master/scripts/review_pr.py
> > > 
> > > This first builds (one or more) packages from src.fpo dist-git forks
> > > that were prepared for PRs in COPR, recursively or non-recursively
> > > queries dependent packages for all of them, and rebuilds them in COPR.
> > > Assuming that there's a copr-cli command for querying build successes,
> > > they could be compared with the latest status of those packages in
> > > koschei, and have it print new build failures. Right now, I compare
> > > the results manually.
> > > 
> > > Would something like this fit your definition of "does-it-blend" script?
> > 
> > What format is '--from-git' in? Say I have a fork with a branch I want
> > to test, what do I pass it?
> 
> --from-git is a regex of package names that are built from dist git instead
> of from latest known SRPM.
> 
> AFAIK The script only supports "the one package" (that is being tested) to
> be built from a custom branch, whoch is the purpose of the script:
> 
> $ python scripts/review_pr.py foobar-2.0 kevin:foobar:branch-2.0
> 
> But you can later built another package in there (with or without depndents)
> in the same copr:
> 
> $ python scripts/review_pr.py foobar-2.0 kevin:bazbaz:fix_deps [-nx'.*']

Cool. The help didn't get me this at all. ;) 

  --from-git FROM_GIT   regexes for packages to build from git

Perhaps that should be "copr-name packager-name:packagename:branchname"
?

Anyhow, after that I ran it on python-dogpile-cache 1.1.1 upgrade:

https://copr.fedorainfracloud.org/coprs/kevin/python-dogpile-cache-1.1.1/packages/

Cool. 

Of course there's still stuff to figure out, like: 
* Since this is a python-package, how many of those actually change on
building against the newer package? I mean, can we tell in a scripted
way how many packages just keep working fine with the new package and
how many need to actually be rebuilt?

* as you noted it would be nice to have some way to say 'package X
failed to build against your change, but it's already failing to build
and this isn't your fault/problem'

It would be nice to modernize the fedora-packager package to include
scripts like this and docs and make them nice. :) 

In any case thanks much for the script and pointers!

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Fabio Valentini]

On Tue, Dec 22, 2020 at 11:16 AM Miro Hrončok  wrote:
>
> On 21. 12. 20 19:36, Kevin Fenzi wrote:
> > On Thu, Dec 17, 2020 at 09:21:15PM +0100, Fabio Valentini wrote:
> >> As Miro mentioned, I've also developed scripts to handle this "does
> >> this update break anything" for the Stewardship / Java SIG, because -
> >> at least at first - we didn't have big enough egos / enough confidence
> >> to just push updates to rawhide without testing excessively them
> >> first:
> >> https://github.com/fedora-stewardship/fedora-stewardship.github.io/blob/master/scripts/review_pr.py
> >>
> >> This first builds (one or more) packages from src.fpo dist-git forks
> >> that were prepared for PRs in COPR, recursively or non-recursively
> >> queries dependent packages for all of them, and rebuilds them in COPR.
> >> Assuming that there's a copr-cli command for querying build successes,
> >> they could be compared with the latest status of those packages in
> >> koschei, and have it print new build failures. Right now, I compare
> >> the results manually.
> >>
> >> Would something like this fit your definition of "does-it-blend" script?
> >
> > What format is '--from-git' in? Say I have a fork with a branch I want
> > to test, what do I pass it?
>
> --from-git is a regex of package names that are built from dist git instead of
> from latest known SRPM.
>
> AFAIK The script only supports "the one package" (that is being tested) to be
> built from a custom branch, whoch is the purpose of the script:
>
> $ python scripts/review_pr.py foobar-2.0 kevin:foobar:branch-2.0
>
> But you can later built another package in there (with or without depndents) 
> in
> the same copr:
>
> $ python scripts/review_pr.py foobar-2.0 kevin:bazbaz:fix_deps [-nx'.*']
>
> > This is indeed the sort of thing I was thinking of (I think).

You should actually be able to build an arbitrary number of packages
before any of their dependencies are rebuilt - just add more
FORKUSERNAME:PKGNAME:BRANCHNAME values. The CLI script explicitly
accepts at-least-one-or-maybe-more (nargs="+") such values.

Side note: I actually did not want to let users specify a PR URL,
since you actually want to test your changes before submitting a PR,
so you can specify the username/package/branch directly without having
to create a PR first. But I think I could add that as an option to the
CLI.

Fabio
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Miro Hrončok]

On 21. 12. 20 19:36, Kevin Fenzi wrote:

On Thu, Dec 17, 2020 at 09:21:15PM +0100, Fabio Valentini wrote:

As Miro mentioned, I've also developed scripts to handle this "does
this update break anything" for the Stewardship / Java SIG, because -
at least at first - we didn't have big enough egos / enough confidence
to just push updates to rawhide without testing excessively them
first:
https://github.com/fedora-stewardship/fedora-stewardship.github.io/blob/master/scripts/review_pr.py

This first builds (one or more) packages from src.fpo dist-git forks
that were prepared for PRs in COPR, recursively or non-recursively
queries dependent packages for all of them, and rebuilds them in COPR.
Assuming that there's a copr-cli command for querying build successes,
they could be compared with the latest status of those packages in
koschei, and have it print new build failures. Right now, I compare
the results manually.

Would something like this fit your definition of "does-it-blend" script?


What format is '--from-git' in? Say I have a fork with a branch I want
to test, what do I pass it?


--from-git is a regex of package names that are built from dist git instead of 
from latest known SRPM.


AFAIK The script only supports "the one package" (that is being tested) to be 
built from a custom branch, whoch is the purpose of the script:


$ python scripts/review_pr.py foobar-2.0 kevin:foobar:branch-2.0

But you can later built another package in there (with or without depndents) in 
the same copr:


$ python scripts/review_pr.py foobar-2.0 kevin:bazbaz:fix_deps [-nx'.*']


This is indeed the sort of thing I was thinking of (I think).



--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Kevin Fenzi]

On Thu, Dec 17, 2020 at 09:21:15PM +0100, Fabio Valentini wrote:
> As Miro mentioned, I've also developed scripts to handle this "does
> this update break anything" for the Stewardship / Java SIG, because -
> at least at first - we didn't have big enough egos / enough confidence
> to just push updates to rawhide without testing excessively them
> first:
> https://github.com/fedora-stewardship/fedora-stewardship.github.io/blob/master/scripts/review_pr.py
> 
> This first builds (one or more) packages from src.fpo dist-git forks
> that were prepared for PRs in COPR, recursively or non-recursively
> queries dependent packages for all of them, and rebuilds them in COPR.
> Assuming that there's a copr-cli command for querying build successes,
> they could be compared with the latest status of those packages in
> koschei, and have it print new build failures. Right now, I compare
> the results manually.
> 
> Would something like this fit your definition of "does-it-blend" script?

What format is '--from-git' in? Say I have a fork with a branch I want
to test, what do I pass it?

This is indeed the sort of thing I was thinking of (I think). 

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Miroslav Suchý]

Dne 19. 12. 20 v 21:24 Luya Tshimbalanga napsal(a):
> Just an idea of a form application that generates the spec file for newcomers 
> as an example.

Something like
  https://xsuchy.github.io/rpm-spec-wizard/
?

I do not propagate it too much as the first page needs a lot of love - it seems 
empty, but there is button in upper left
corner, which does the magic.

-- 
Miroslav Suchy, RHCA
Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Mikolaj Izdebski]

On Mon, Dec 21, 2020 at 11:13 AM Vít Ondruch  wrote:
> > There is also a possibility for a group or individual to request their
> > packages to be auto-tracked - whenever the person or group gets at
> > least commit privileges to a package in dist-git, Koschei will start
> > tracking it. To request auto-tracking of packages in Koschei you can
> > open a ticket with Fedora Infrastructure, or contact me directly.
>
>
> I definitely want all my packages autotracked! It happens from time to
> time I adopt some package just to find out later it is not included in
> Koschei. And similar case are probably new packages ...

Auto-tracking for individual users is currently broken due to Pagure
API break [1], so it needs development work to fix it.

[1] https://pagure.io/pagure/issue/3824

>
>
> Vít
>
>
> >
> > --
> > Mikolaj Izdebski
> > ___
> > devel mailing list -- devel@lists.fedoraproject.org
> > To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> > Fedora Code of Conduct: 
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives: 
> > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Vít Ondruch]

Dne 21. 12. 20 v 10:37 Mikolaj Izdebski napsal(a):

On Wed, Dec 16, 2020 at 8:18 PM Jonathan Wakely
 wrote:

Can we just add everything to Koschei instead
of having it opt-in?



+1

It happens quite often that I stumble upon some package which is already 
FTBFS, but it is hard to tell for how long and it is even harder to know 
what is the reason. Koschei would help here.




That is technically possible, but so far no one asked about this.
Currently 53 % of all packages (and 59 % of rawhide packages) are
"tracked", which means Koschei submits scratch builds for them. The
rest is not tracked, which means Koschei only resolves dependencies
for them, but does not submit scratch builds to Koji to save
resources.

There is also a possibility for a group or individual to request their
packages to be auto-tracked - whenever the person or group gets at
least commit privileges to a package in dist-git, Koschei will start
tracking it. To request auto-tracking of packages in Koschei you can
open a ticket with Fedora Infrastructure, or contact me directly.



I definitely want all my packages autotracked! It happens from time to 
time I adopt some package just to find out later it is not included in 
Koschei. And similar case are probably new packages ...



Vít




--
Mikolaj Izdebski
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


OpenPGP_0x0CE09EE79917B87C.asc
Description: application/pgp-keys


OpenPGP_signature
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Vít Ondruch]

Dne 17. 12. 20 v 21:17 Kevin Fenzi napsal(a):

On Thu, Dec 17, 2020 at 03:26:47PM +0100, Vít Ondruch wrote:

Sadly we do not have the amount of disk space needed for that. Koschei
fills up our koji partition regularly. The partition has a 100TB HARD
limit and between koschei, ELN and regular update rebuilds we bounce off
it multiple times a release. Then when we try to garbage collect all
those we regularly end up with koji outages.. which then pile up a whole
bunch of new ones which have to be garbage collected.

For longer koschei builds we would need to do this in a different
resources than what we have.


I understand, but if the garbage collection initially removed everything
except logs, that would help tremendously.

Huh. It's supposed to be working this way now.



Ah, I was not aware of this (because I typically don't care about the 
build artifacts except logs), so thx for clarification.



  


After 1 day the rpms are removed.
After 7 days the logs are removed.

Non koschei scratch builds (both logs and rpms) are kept for 14 days.

Would keeping the logs for 14 days help?



That would be definitely appreciated.

The issues is the Koschei does not build every package all the time. 
There might be bigger time span between two subsequent Koschei builds. 
Therefore once there is new build failure, the old logs might be gone 
already. This of course makes harder to compare root.logs to see what 
have changed in buildroot (I know the Koschei lists package changes, but 
there are unfortunately not reliable).



Vít



Or is it not somehow doing that?

kevin

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


OpenPGP_0x0CE09EE79917B87C.asc
Description: application/pgp-keys


OpenPGP_signature
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Mikolaj Izdebski]

On Wed, Dec 16, 2020 at 8:18 PM Jonathan Wakely
 wrote:
> Can we just add everything to Koschei instead
> of having it opt-in?

That is technically possible, but so far no one asked about this.
Currently 53 % of all packages (and 59 % of rawhide packages) are
"tracked", which means Koschei submits scratch builds for them. The
rest is not tracked, which means Koschei only resolves dependencies
for them, but does not submit scratch builds to Koji to save
resources.

There is also a possibility for a group or individual to request their
packages to be auto-tracked - whenever the person or group gets at
least commit privileges to a package in dist-git, Koschei will start
tracking it. To request auto-tracking of packages in Koschei you can
open a ticket with Fedora Infrastructure, or contact me directly.

--
Mikolaj Izdebski
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Luya Tshimbalanga]

On 2020-12-17 3:27 a.m., Miroslav Suchý wrote:

Dne 17. 12. 20 v 8:02 Luya Tshimbalanga napsal(a):

Frontend for spec file

??? Can you elaborate it?

Just an idea of a form application that generates the spec file for 
newcomers as an example.


--
Luya Tshimbalanga
Fedora Design Team
Fedora Design Suite maintainer
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Dan Čermák]

Jonathan Wakely  writes:

> On 15/12/20 23:46 +0100, Miro Hrončok wrote:
>>On 12/15/20 11:29 PM, Miroslav Suchý wrote:
>>>I am looking for challenges for upcoming year - what I and my team should 
>>>enhance. I have some ideas, but I want to hear
>>>yours.
>>
>>Thanks for doing this!
>>
>>>What you - as Fedora packager - find most time consuming on packaging?
>>
>>Coordination and communication ;)
>>
>>>Where you will welcome more simplicity or automation?
>>
>>In testing an impact of a change. E.g. a simple "upgrade to newer 
>>version" change might be a problem if it breaks other packages. I 
>>usually spin up a copr repo and try to rebuild every dependent package 
>
> ^ This.
>
> The individual steps of doing a single package are insignificant
> compared to the days or WEEKS it takes for a systemwide change that
> requires rebuilding hundreds of dependencies. I know not many of us
> actually have this problem, but for those who do, it's very, very time
> consuming.
>
> Since we're apparently not allowed to use a side tag to do a test
> rebuild for this kind of thing, I end up doing it locally on my own
> machines. Copr is another option, but I don't think it would be any
> quicker or simpler.
>
> What I'd really like would be a "test mass rebuild" process, where a
> prospective package is uploaded and everything that depends on it is
> automatically rebuilt (ideally after creating the dependency graph so
> each individual package doesn't get rebuilt until its dependencies are
> finished building).
>
> Creating the dependency graph by hand is fairly tedious, but maybe I'm
> missing an automated way. The point of creating that graph is to avoid
> wasting time and power doing and redoing builds that will fail until
> something else has been built (which is the problem with mock's
> --chain command, if I understand correctly).
>
> Once I have that graph, I use Make to control the process, because it
> handles the dependency graph, as well as parallelism, and not
> rebuilding things unnecessarily.

Assuming that you have all packages that you care about checked out in a
directory, you can use Jens Petersen's fbrnch to build them in the
correct order: https://github.com/juhp/fbrnch#parallel-building
But, that only works if there are no dependency loops in your packages,
which is more or less guaranteed in a non-trivial subset set of all
Fedora packages.

Iirc there was also the idea to leverage Zuul for doing these kinds of
automated tests, but I am not sure how feasible that actually is. Even
in openSUSE Tumbleweed, which is developed in OBS, automated rebuilds
are disabled and only a small set of all packages is rebuild in a
staging area before being other packages are merged into the distro. And
that's for the simple reason, that OBS does not have enough workers to
support this (and OBS has more workers than we do, while Tumbleweed has
less packages than Fedora).

>
>>in it. However, there are time consuming challenges with this:
>>
>>1) False failures. Sometimes, the copr build fails for random reason 
>>(Koji repo is not available, etc.). I need to read the logs and figure 
>>it out, resubmit the build.
>>
>>2) Unrelated failures. Many packages FTBFS for unrelated reasons. I 
>>need to spin up another (control) copr and rebuild all failures there 
>>as well to make sure it's indeed unrelated.
>
> Yes, that's a real pain. Can we just add everything to Koschei instead
> of having it opt-in?

Yes please, Koschei should be opt-out, not opt-in.



signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Fabio Valentini]

On Thu, Dec 17, 2020 at 9:08 PM Kevin Fenzi  wrote:
>
> On Wed, Dec 16, 2020 at 03:56:25PM -0500, Neal Gompa wrote:
> ...snip...
> > So I've written tools for doing this, and Igor has written tools for
> > doing this, but it seems like people think that this is "impossible"
> > and so the effort goes nowhere despite several PoCs.
> >
> > If we're interested in this again for real this time, I could try to
> > dig out my old code for it, but we might be better off just pulling
> > out Koschei's code and turning it into something that Koji's
> > chain-build command and Mock's --chain option use to sort through
> > package sets and build them correctly.
>
> Can you expand on which 'this' you mean? Getting the build
> order/dependency graph? Or a tool to use that to rebuild everything and
> tell you what failed? or ?
>
> I don't think you can ever be 100% on dependency graph/build order,
> because there's sometimes bootstraps or loops in there. :(
>
> Anyhow I would love to be able to locally build a new library and run
> 'fedpkg does-it-blend foo.src.rpm' and have it tell me exaclty what
> other packages that breaks.

As Miro mentioned, I've also developed scripts to handle this "does
this update break anything" for the Stewardship / Java SIG, because -
at least at first - we didn't have big enough egos / enough confidence
to just push updates to rawhide without testing excessively them
first:
https://github.com/fedora-stewardship/fedora-stewardship.github.io/blob/master/scripts/review_pr.py

This first builds (one or more) packages from src.fpo dist-git forks
that were prepared for PRs in COPR, recursively or non-recursively
queries dependent packages for all of them, and rebuilds them in COPR.
Assuming that there's a copr-cli command for querying build successes,
they could be compared with the latest status of those packages in
koschei, and have it print new build failures. Right now, I compare
the results manually.

Would something like this fit your definition of "does-it-blend" script?

Fabio
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Kevin Fenzi]

On Thu, Dec 17, 2020 at 03:26:47PM +0100, Vít Ondruch wrote:
> > Sadly we do not have the amount of disk space needed for that. Koschei
> > fills up our koji partition regularly. The partition has a 100TB HARD
> > limit and between koschei, ELN and regular update rebuilds we bounce off
> > it multiple times a release. Then when we try to garbage collect all
> > those we regularly end up with koji outages.. which then pile up a whole
> > bunch of new ones which have to be garbage collected.
> > 
> > For longer koschei builds we would need to do this in a different
> > resources than what we have.
> 
> 
> I understand, but if the garbage collection initially removed everything
> except logs, that would help tremendously.

Huh. It's supposed to be working this way now. 

After 1 day the rpms are removed. 
After 7 days the logs are removed. 

Non koschei scratch builds (both logs and rpms) are kept for 14 days. 

Would keeping the logs for 14 days help?
Or is it not somehow doing that?

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Kevin Fenzi]

On Wed, Dec 16, 2020 at 03:56:25PM -0500, Neal Gompa wrote:
...snip...
> > > What I'd really like would be a "test mass rebuild" process, where a
> > > prospective package is uploaded and everything that depends on it is
> > > automatically rebuilt (ideally after creating the dependency graph so
> > > each individual package doesn't get rebuilt until its dependencies are
> > > finished building).
> > >
> > > Creating the dependency graph by hand is fairly tedious, but maybe I'm
> > > missing an automated way. The point of creating that graph is to avoid
> > > wasting time and power doing and redoing builds that will fail until
> > > something else has been built (which is the problem with mock's
> > > --chain command, if I understand correctly).
> > >
> > > Once I have that graph, I use Make to control the process, because it
> > > handles the dependency graph, as well as parallelism, and not
> > > rebuilding things unnecessarily.
> >
> > Yeah, all this ^
> >
> 
> So I've written tools for doing this, and Igor has written tools for
> doing this, but it seems like people think that this is "impossible"
> and so the effort goes nowhere despite several PoCs.
> 
> If we're interested in this again for real this time, I could try to
> dig out my old code for it, but we might be better off just pulling
> out Koschei's code and turning it into something that Koji's
> chain-build command and Mock's --chain option use to sort through
> package sets and build them correctly.

Can you expand on which 'this' you mean? Getting the build
order/dependency graph? Or a tool to use that to rebuild everything and
tell you what failed? or ?

I don't think you can ever be 100% on dependency graph/build order,
because there's sometimes bootstraps or loops in there. :(

Anyhow I would love to be able to locally build a new library and run
'fedpkg does-it-blend foo.src.rpm' and have it tell me exaclty what
other packages that breaks. 

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Vít Ondruch]

Dne 17. 12. 20 v 15:06 Stephen John Smoogen napsal(a):



On Thu, 17 Dec 2020 at 04:14, Vít Ondruch > wrote:



Dne 16. 12. 20 v 21:56 Neal Gompa napsal(a):

>>> Yes, that's a real pain. Can we just add everything to Koschei
instead
>>> of having it opt-in?
>> Thats worth considering in the new year. I would like that. :)
>>

I would appreciate if Koschei kept more results. It is almost
unuseful
these days if one does not have time to check the issue right
after it
appears :(



Sadly we do not have the amount of disk space needed for that. Koschei 
fills up our koji partition regularly. The partition has a 100TB HARD 
limit and between koschei, ELN and regular update rebuilds we bounce 
off it multiple times a release. Then when we try to garbage collect 
all those we regularly end up with koji outages.. which then pile up a 
whole bunch of new ones which have to be garbage collected.


For longer koschei builds we would need to do this in a different 
resources than what we have.



I understand, but if the garbage collection initially removed everything 
except logs, that would help tremendously.



Vít



Vít


> Yes please!
>
>
>
>
> --
> 真実はいつも一つ！/ Always, there's only one truth!
> ___
> devel mailing list -- devel@lists.fedoraproject.org

> To unsubscribe send an email to
devel-le...@lists.fedoraproject.org

> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/

> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines

> List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


___
devel mailing list -- devel@lists.fedoraproject.org

To unsubscribe send an email to
devel-le...@lists.fedoraproject.org

Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/

List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines

List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org





--
Stephen J Smoogen.


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


OpenPGP_0x0CE09EE79917B87C.asc
Description: application/pgp-keys


OpenPGP_signature
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Stephen John Smoogen]

On Thu, 17 Dec 2020 at 04:14, Vít Ondruch  wrote:

>
> Dne 16. 12. 20 v 21:56 Neal Gompa napsal(a):
>
> >>> Yes, that's a real pain. Can we just add everything to Koschei instead
> >>> of having it opt-in?
> >> Thats worth considering in the new year. I would like that. :)
> >>
>
> I would appreciate if Koschei kept more results. It is almost unuseful
> these days if one does not have time to check the issue right after it
> appears :(
>
>
>
Sadly we do not have the amount of disk space needed for that. Koschei
fills up our koji partition regularly. The partition has a 100TB HARD limit
and between koschei, ELN and regular update rebuilds we bounce off it
multiple times a release. Then when we try to garbage collect all those we
regularly end up with koji outages.. which then pile up a whole bunch of
new ones which have to be garbage collected.

For longer koschei builds we would need to do this in a different resources
than what we have.


> Vít
>
>
> > Yes please!
> >
> >
> >
> >
> > --
> > 真実はいつも一つ！/ Always, there's only one truth!
> > ___
> > devel mailing list -- devel@lists.fedoraproject.org
> > To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> > Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>


-- 
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Vitaly Zaitsev via devel]

On 16.12.2020 20:15, Jonathan Wakely wrote:

What I'd really like would be a "test mass rebuild" process, where a
prospective package is uploaded and everything that depends on it is
automatically rebuilt (ideally after creating the dependency graph so
each individual package doesn't get rebuilt until its dependencies are
finished building).


OBS supports this feature.

--
Sincerely,
  Vitaly Zaitsev (vit...@easycoding.org)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Vít Ondruch]

Dne 17. 12. 20 v 12:30 Fabio Valentini napsal(a):

On Thu, Dec 17, 2020 at 10:14 AM Vít Ondruch  wrote:


Dne 16. 12. 20 v 21:56 Neal Gompa napsal(a):

On Wed, Dec 16, 2020 at 3:34 PM Kevin Fenzi  wrote:

On Wed, Dec 16, 2020 at 07:15:21PM +, Jonathan Wakely wrote:

On 15/12/20 23:46 +0100, Miro Hrončok wrote:

On 12/15/20 11:29 PM, Miroslav Suchý wrote:

I am looking for challenges for upcoming year - what I and my team should 
enhance. I have some ideas, but I want to hear
yours.

Thanks for doing this!


What you - as Fedora packager - find most time consuming on packaging?

Coordination and communication ;)


Where you will welcome more simplicity or automation?

In testing an impact of a change. E.g. a simple "upgrade to newer
version" change might be a problem if it breaks other packages. I
usually spin up a copr repo and try to rebuild every dependent package

^ This.

The individual steps of doing a single package are insignificant
compared to the days or WEEKS it takes for a systemwide change that
requires rebuilding hundreds of dependencies. I know not many of us
actually have this problem, but for those who do, it's very, very time
consuming.

Since we're apparently not allowed to use a side tag to do a test
rebuild for this kind of thing, I end up doing it locally on my own
machines. Copr is another option, but I don't think it would be any
quicker or simpler.

You actually can use a side tag, but it's not very streamlined.
1. make side-tag
2. push your changes to a whatever branch on the package that you are
changing.
3. build package from that branch into the sidetag
4. scratch build all the dependent packages in the sidetag and they will
use the changed package. Get them all working.
5. merge your branch back on the package, bump release again
5. actually bump and officially rebuild all the packages in the side tag
6. merge sidetag
7. ask releng to delete the branch you made (or not if you don't care)

But I agree thats not great. ;(


What I'd really like would be a "test mass rebuild" process, where a
prospective package is uploaded and everything that depends on it is
automatically rebuilt (ideally after creating the dependency graph so
each individual package doesn't get rebuilt until its dependencies are
finished building).

Creating the dependency graph by hand is fairly tedious, but maybe I'm
missing an automated way. The point of creating that graph is to avoid
wasting time and power doing and redoing builds that will fail until
something else has been built (which is the problem with mock's
--chain command, if I understand correctly).

Once I have that graph, I use Make to control the process, because it
handles the dependency graph, as well as parallelism, and not
rebuilding things unnecessarily.

Yeah, all this ^


So I've written tools for doing this, and Igor has written tools for
doing this, but it seems like people think that this is "impossible"
and so the effort goes nowhere despite several PoCs.

If we're interested in this again for real this time, I could try to
dig out my old code for it, but we might be better off just pulling
out Koschei's code and turning it into something that Koji's
chain-build command and Mock's --chain option use to sort through
package sets and build them correctly.


in it. However, there are time consuming challenges with this:

1) False failures. Sometimes, the copr build fails for random reason
(Koji repo is not available, etc.). I need to read the logs and figure
it out, resubmit the build.

2) Unrelated failures. Many packages FTBFS for unrelated reasons. I need
to spin up another (control) copr and rebuild all failures there as well
to make sure it's indeed unrelated.

Yes, that's a real pain. Can we just add everything to Koschei instead
of having it opt-in?

Thats worth considering in the new year. I would like that. :)


I would appreciate if Koschei kept more results. It is almost unuseful
these days if one does not have time to check the issue right after it
appears :(

I think this is caused by koji garbage collection?
I assume the retention time it's set very short for scratch builds,
which makes koschei less useful, but keeps storage use in check ...



The root cause is lack of storage :/

Actually I made a RFE to keep copies on Koschei side, but it has not 
materialized:


https://github.com/fedora-infra/koschei/issues/123

(and it might just shift the issue elsewhere, while the 
root.log/build.log would be probably enough and would not consume so 
much space as every build artefact).



V.




Fabio
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Fabio Valentini]

On Thu, Dec 17, 2020 at 10:14 AM Vít Ondruch  wrote:
>
>
> Dne 16. 12. 20 v 21:56 Neal Gompa napsal(a):
> > On Wed, Dec 16, 2020 at 3:34 PM Kevin Fenzi  wrote:
> >> On Wed, Dec 16, 2020 at 07:15:21PM +, Jonathan Wakely wrote:
> >>> On 15/12/20 23:46 +0100, Miro Hrončok wrote:
>  On 12/15/20 11:29 PM, Miroslav Suchý wrote:
> > I am looking for challenges for upcoming year - what I and my team 
> > should enhance. I have some ideas, but I want to hear
> > yours.
>  Thanks for doing this!
> 
> > What you - as Fedora packager - find most time consuming on packaging?
>  Coordination and communication ;)
> 
> > Where you will welcome more simplicity or automation?
>  In testing an impact of a change. E.g. a simple "upgrade to newer
>  version" change might be a problem if it breaks other packages. I
>  usually spin up a copr repo and try to rebuild every dependent package
> >>> ^ This.
> >>>
> >>> The individual steps of doing a single package are insignificant
> >>> compared to the days or WEEKS it takes for a systemwide change that
> >>> requires rebuilding hundreds of dependencies. I know not many of us
> >>> actually have this problem, but for those who do, it's very, very time
> >>> consuming.
> >>>
> >>> Since we're apparently not allowed to use a side tag to do a test
> >>> rebuild for this kind of thing, I end up doing it locally on my own
> >>> machines. Copr is another option, but I don't think it would be any
> >>> quicker or simpler.
> >> You actually can use a side tag, but it's not very streamlined.
> >> 1. make side-tag
> >> 2. push your changes to a whatever branch on the package that you are
> >> changing.
> >> 3. build package from that branch into the sidetag
> >> 4. scratch build all the dependent packages in the sidetag and they will
> >> use the changed package. Get them all working.
> >> 5. merge your branch back on the package, bump release again
> >> 5. actually bump and officially rebuild all the packages in the side tag
> >> 6. merge sidetag
> >> 7. ask releng to delete the branch you made (or not if you don't care)
> >>
> >> But I agree thats not great. ;(
> >>
> >>> What I'd really like would be a "test mass rebuild" process, where a
> >>> prospective package is uploaded and everything that depends on it is
> >>> automatically rebuilt (ideally after creating the dependency graph so
> >>> each individual package doesn't get rebuilt until its dependencies are
> >>> finished building).
> >>>
> >>> Creating the dependency graph by hand is fairly tedious, but maybe I'm
> >>> missing an automated way. The point of creating that graph is to avoid
> >>> wasting time and power doing and redoing builds that will fail until
> >>> something else has been built (which is the problem with mock's
> >>> --chain command, if I understand correctly).
> >>>
> >>> Once I have that graph, I use Make to control the process, because it
> >>> handles the dependency graph, as well as parallelism, and not
> >>> rebuilding things unnecessarily.
> >> Yeah, all this ^
> >>
> > So I've written tools for doing this, and Igor has written tools for
> > doing this, but it seems like people think that this is "impossible"
> > and so the effort goes nowhere despite several PoCs.
> >
> > If we're interested in this again for real this time, I could try to
> > dig out my old code for it, but we might be better off just pulling
> > out Koschei's code and turning it into something that Koji's
> > chain-build command and Mock's --chain option use to sort through
> > package sets and build them correctly.
> >
>  in it. However, there are time consuming challenges with this:
> 
>  1) False failures. Sometimes, the copr build fails for random reason
>  (Koji repo is not available, etc.). I need to read the logs and figure
>  it out, resubmit the build.
> 
>  2) Unrelated failures. Many packages FTBFS for unrelated reasons. I need
>  to spin up another (control) copr and rebuild all failures there as well
>  to make sure it's indeed unrelated.
> >>> Yes, that's a real pain. Can we just add everything to Koschei instead
> >>> of having it opt-in?
> >> Thats worth considering in the new year. I would like that. :)
> >>
>
> I would appreciate if Koschei kept more results. It is almost unuseful
> these days if one does not have time to check the issue right after it
> appears :(

I think this is caused by koji garbage collection?
I assume the retention time it's set very short for scratch builds,
which makes koschei less useful, but keeps storage use in check ...

Fabio
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 

Re: What is the most time consuming task for you as packager?

[Fabio Valentini]

On Thu, Dec 17, 2020 at 1:02 AM Chuck Anderson  wrote:
>
> On Tue, Dec 15, 2020 at 11:29:27PM +0100, Miroslav Suchý wrote:
> > I am looking for challenges for upcoming year - what I and my team should 
> > enhance. I have some ideas, but I want to hear
> > yours.
> >
> > What you - as Fedora packager - find most time consuming on packaging?
> > Where you will welcome more simplicity or automation?
>
> Looking up the latest guidelines and recommendations for packaging.
> It seems to be spread between the Wiki and "docs" site and it isn't
> always clear which one is correct.  Google searches may still find old
> or "draft" guidelines.  Not all are ported over to docs.fp.org, so you
> have to use the Wiki site for some content.  Wiki documents link to
> themselves, even when in some cases the Wiki version is out of date,
> replaced by docs.fp.org version.  There isn't always a warning or
> reminder that links to the correct version on docs.fp.org.  Or there
> is a redirect to the new site, but it doesn't link to the right
> content.
>
> https://fedoraproject.org/wiki/Packaging:Guidelines
>
> "The packaging guidelines have been moved out of the wiki. The current
> version of this document is located at
> https://docs.fedoraproject.org/en-US/packaging-guidelines/ . Please
> update your bookmarks."
>
> But say I search Google for "fedora perl packaging guidelines" I find this:
>
> https://fedoraproject.org/wiki/Packaging:Perl
>
> Then there is a link on there to:
>
> https://fedoraproject.org/wiki/Category:Packaging_guidelines
>
> Or search Google for "fedora packaging systemctl" finds this:
>
> https://fedoraproject.org/wiki/Packaging:Systemd
>
> which has:
>
> Unit files in spec file scriptlets
>
> Information on proper handling of unit files in spec file scriptlets can be 
> found here: Packaging:Scriptlets#Systemd
>
> but that links to:
>
> https://fedoraproject.org/wiki/Packaging:Scriptlets#Systemd
>
> which redirects to:
>
> https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/#Systemd
>
> which doesn't have a #Systemd anchor.  So you are dumped at the top of
> that page rather than at the section about Systemd which has an anchor
> of #_systemd instead.
>
> It would be nice if this could all be cleaned up, because right now it
> makes navigating the documentation a nightmare.

We (the Packaging Committee) know about this problem, and we've been
slowly working to get the migrated pages "verified" and to add
redirects from the old Wiki pages to the corresponding new docs.
However, this is a very time-consuming process (checking that all the
content has been migrated correctly, checking links, checking
formatting, etc), and it has somewhat stalled recently, because, ya
know, 2020.

https://pagure.io/packaging-committee/issue/845

Fabio
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Miroslav Suchý]

Dne 17. 12. 20 v 8:02 Luya Tshimbalanga napsal(a):
> Frontend for spec file

??? Can you elaborate it?

-- 
Miroslav Suchy, RHCA
Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Vít Ondruch]

Dne 16. 12. 20 v 21:56 Neal Gompa napsal(a):

On Wed, Dec 16, 2020 at 3:34 PM Kevin Fenzi  wrote:

On Wed, Dec 16, 2020 at 07:15:21PM +, Jonathan Wakely wrote:

On 15/12/20 23:46 +0100, Miro Hrončok wrote:

On 12/15/20 11:29 PM, Miroslav Suchý wrote:

I am looking for challenges for upcoming year - what I and my team should 
enhance. I have some ideas, but I want to hear
yours.

Thanks for doing this!


What you - as Fedora packager - find most time consuming on packaging?

Coordination and communication ;)


Where you will welcome more simplicity or automation?

In testing an impact of a change. E.g. a simple "upgrade to newer
version" change might be a problem if it breaks other packages. I
usually spin up a copr repo and try to rebuild every dependent package

^ This.

The individual steps of doing a single package are insignificant
compared to the days or WEEKS it takes for a systemwide change that
requires rebuilding hundreds of dependencies. I know not many of us
actually have this problem, but for those who do, it's very, very time
consuming.

Since we're apparently not allowed to use a side tag to do a test
rebuild for this kind of thing, I end up doing it locally on my own
machines. Copr is another option, but I don't think it would be any
quicker or simpler.

You actually can use a side tag, but it's not very streamlined.
1. make side-tag
2. push your changes to a whatever branch on the package that you are
changing.
3. build package from that branch into the sidetag
4. scratch build all the dependent packages in the sidetag and they will
use the changed package. Get them all working.
5. merge your branch back on the package, bump release again
5. actually bump and officially rebuild all the packages in the side tag
6. merge sidetag
7. ask releng to delete the branch you made (or not if you don't care)

But I agree thats not great. ;(


What I'd really like would be a "test mass rebuild" process, where a
prospective package is uploaded and everything that depends on it is
automatically rebuilt (ideally after creating the dependency graph so
each individual package doesn't get rebuilt until its dependencies are
finished building).

Creating the dependency graph by hand is fairly tedious, but maybe I'm
missing an automated way. The point of creating that graph is to avoid
wasting time and power doing and redoing builds that will fail until
something else has been built (which is the problem with mock's
--chain command, if I understand correctly).

Once I have that graph, I use Make to control the process, because it
handles the dependency graph, as well as parallelism, and not
rebuilding things unnecessarily.

Yeah, all this ^


So I've written tools for doing this, and Igor has written tools for
doing this, but it seems like people think that this is "impossible"
and so the effort goes nowhere despite several PoCs.

If we're interested in this again for real this time, I could try to
dig out my old code for it, but we might be better off just pulling
out Koschei's code and turning it into something that Koji's
chain-build command and Mock's --chain option use to sort through
package sets and build them correctly.


in it. However, there are time consuming challenges with this:

1) False failures. Sometimes, the copr build fails for random reason
(Koji repo is not available, etc.). I need to read the logs and figure
it out, resubmit the build.

2) Unrelated failures. Many packages FTBFS for unrelated reasons. I need
to spin up another (control) copr and rebuild all failures there as well
to make sure it's indeed unrelated.

Yes, that's a real pain. Can we just add everything to Koschei instead
of having it opt-in?

Thats worth considering in the new year. I would like that. :)



I would appreciate if Koschei kept more results. It is almost unuseful 
these days if one does not have time to check the issue right after it 
appears :(



Vít



Yes please!




--
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


OpenPGP_0x0CE09EE79917B87C.asc
Description: application/pgp-keys


OpenPGP_signature
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Luya Tshimbalanga]

On 2020-12-15 2:29 p.m., Miroslav Suchý wrote:

I am looking for challenges for upcoming year - what I and my team should 
enhance. I have some ideas, but I want to hear
yours.

What you - as Fedora packager - find most time consuming on packaging?


 * Communication and synchronization with a dependent packages.
 * Outdated spec templates.


Where you will welcome more simplicity or automation?


 * Majority of packages I maintained are linked with release-monitoring
   and will be great the tools simply automatically updates package
   should the scratch build succeed.
 * Frontend for spec file



--
Luya Tshimbalanga
Fedora Design Team
Fedora Design Suite maintainer

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Chuck Anderson]

On Tue, Dec 15, 2020 at 11:29:27PM +0100, Miroslav Suchý wrote:
> I am looking for challenges for upcoming year - what I and my team should 
> enhance. I have some ideas, but I want to hear
> yours.
> 
> What you - as Fedora packager - find most time consuming on packaging?
> Where you will welcome more simplicity or automation?

Looking up the latest guidelines and recommendations for packaging.
It seems to be spread between the Wiki and "docs" site and it isn't
always clear which one is correct.  Google searches may still find old
or "draft" guidelines.  Not all are ported over to docs.fp.org, so you
have to use the Wiki site for some content.  Wiki documents link to
themselves, even when in some cases the Wiki version is out of date,
replaced by docs.fp.org version.  There isn't always a warning or
reminder that links to the correct version on docs.fp.org.  Or there
is a redirect to the new site, but it doesn't link to the right
content.

https://fedoraproject.org/wiki/Packaging:Guidelines

"The packaging guidelines have been moved out of the wiki. The current
version of this document is located at
https://docs.fedoraproject.org/en-US/packaging-guidelines/ . Please
update your bookmarks."

But say I search Google for "fedora perl packaging guidelines" I find this:

https://fedoraproject.org/wiki/Packaging:Perl

Then there is a link on there to:

https://fedoraproject.org/wiki/Category:Packaging_guidelines

Or search Google for "fedora packaging systemctl" finds this:

https://fedoraproject.org/wiki/Packaging:Systemd

which has:

 Unit files in spec file scriptlets

Information on proper handling of unit files in spec file scriptlets can be 
found here: Packaging:Scriptlets#Systemd

but that links to:

https://fedoraproject.org/wiki/Packaging:Scriptlets#Systemd

which redirects to:

https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/#Systemd

which doesn't have a #Systemd anchor.  So you are dumped at the top of
that page rather than at the section about Systemd which has an anchor
of #_systemd instead.

It would be nice if this could all be cleaned up, because right now it
makes navigating the documentation a nightmare.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Neal Gompa]

On Wed, Dec 16, 2020 at 3:34 PM Kevin Fenzi  wrote:
>
> On Wed, Dec 16, 2020 at 07:15:21PM +, Jonathan Wakely wrote:
> > On 15/12/20 23:46 +0100, Miro Hrončok wrote:
> > > On 12/15/20 11:29 PM, Miroslav Suchý wrote:
> > > > I am looking for challenges for upcoming year - what I and my team 
> > > > should enhance. I have some ideas, but I want to hear
> > > > yours.
> > >
> > > Thanks for doing this!
> > >
> > > > What you - as Fedora packager - find most time consuming on packaging?
> > >
> > > Coordination and communication ;)
> > >
> > > > Where you will welcome more simplicity or automation?
> > >
> > > In testing an impact of a change. E.g. a simple "upgrade to newer
> > > version" change might be a problem if it breaks other packages. I
> > > usually spin up a copr repo and try to rebuild every dependent package
> >
> > ^ This.
> >
> > The individual steps of doing a single package are insignificant
> > compared to the days or WEEKS it takes for a systemwide change that
> > requires rebuilding hundreds of dependencies. I know not many of us
> > actually have this problem, but for those who do, it's very, very time
> > consuming.
> >
> > Since we're apparently not allowed to use a side tag to do a test
> > rebuild for this kind of thing, I end up doing it locally on my own
> > machines. Copr is another option, but I don't think it would be any
> > quicker or simpler.
>
> You actually can use a side tag, but it's not very streamlined.
> 1. make side-tag
> 2. push your changes to a whatever branch on the package that you are
> changing.
> 3. build package from that branch into the sidetag
> 4. scratch build all the dependent packages in the sidetag and they will
> use the changed package. Get them all working.
> 5. merge your branch back on the package, bump release again
> 5. actually bump and officially rebuild all the packages in the side tag
> 6. merge sidetag
> 7. ask releng to delete the branch you made (or not if you don't care)
>
> But I agree thats not great. ;(
>
> > What I'd really like would be a "test mass rebuild" process, where a
> > prospective package is uploaded and everything that depends on it is
> > automatically rebuilt (ideally after creating the dependency graph so
> > each individual package doesn't get rebuilt until its dependencies are
> > finished building).
> >
> > Creating the dependency graph by hand is fairly tedious, but maybe I'm
> > missing an automated way. The point of creating that graph is to avoid
> > wasting time and power doing and redoing builds that will fail until
> > something else has been built (which is the problem with mock's
> > --chain command, if I understand correctly).
> >
> > Once I have that graph, I use Make to control the process, because it
> > handles the dependency graph, as well as parallelism, and not
> > rebuilding things unnecessarily.
>
> Yeah, all this ^
>

So I've written tools for doing this, and Igor has written tools for
doing this, but it seems like people think that this is "impossible"
and so the effort goes nowhere despite several PoCs.

If we're interested in this again for real this time, I could try to
dig out my old code for it, but we might be better off just pulling
out Koschei's code and turning it into something that Koji's
chain-build command and Mock's --chain option use to sort through
package sets and build them correctly.

> > > in it. However, there are time consuming challenges with this:
> > >
> > > 1) False failures. Sometimes, the copr build fails for random reason
> > > (Koji repo is not available, etc.). I need to read the logs and figure
> > > it out, resubmit the build.
> > >
> > > 2) Unrelated failures. Many packages FTBFS for unrelated reasons. I need
> > > to spin up another (control) copr and rebuild all failures there as well
> > > to make sure it's indeed unrelated.
> >
> > Yes, that's a real pain. Can we just add everything to Koschei instead
> > of having it opt-in?
>
> Thats worth considering in the new year. I would like that. :)
>

Yes please!




--
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Kevin Fenzi]

On Wed, Dec 16, 2020 at 07:15:21PM +, Jonathan Wakely wrote:
> On 15/12/20 23:46 +0100, Miro Hrončok wrote:
> > On 12/15/20 11:29 PM, Miroslav Suchý wrote:
> > > I am looking for challenges for upcoming year - what I and my team should 
> > > enhance. I have some ideas, but I want to hear
> > > yours.
> > 
> > Thanks for doing this!
> > 
> > > What you - as Fedora packager - find most time consuming on packaging?
> > 
> > Coordination and communication ;)
> > 
> > > Where you will welcome more simplicity or automation?
> > 
> > In testing an impact of a change. E.g. a simple "upgrade to newer
> > version" change might be a problem if it breaks other packages. I
> > usually spin up a copr repo and try to rebuild every dependent package
> 
> ^ This.
> 
> The individual steps of doing a single package are insignificant
> compared to the days or WEEKS it takes for a systemwide change that
> requires rebuilding hundreds of dependencies. I know not many of us
> actually have this problem, but for those who do, it's very, very time
> consuming.
> 
> Since we're apparently not allowed to use a side tag to do a test
> rebuild for this kind of thing, I end up doing it locally on my own
> machines. Copr is another option, but I don't think it would be any
> quicker or simpler.

You actually can use a side tag, but it's not very streamlined. 
1. make side-tag
2. push your changes to a whatever branch on the package that you are
changing. 
3. build package from that branch into the sidetag
4. scratch build all the dependent packages in the sidetag and they will
use the changed package. Get them all working. 
5. merge your branch back on the package, bump release again
5. actually bump and officially rebuild all the packages in the side tag
6. merge sidetag
7. ask releng to delete the branch you made (or not if you don't care)

But I agree thats not great. ;( 

> What I'd really like would be a "test mass rebuild" process, where a
> prospective package is uploaded and everything that depends on it is
> automatically rebuilt (ideally after creating the dependency graph so
> each individual package doesn't get rebuilt until its dependencies are
> finished building).
> 
> Creating the dependency graph by hand is fairly tedious, but maybe I'm
> missing an automated way. The point of creating that graph is to avoid
> wasting time and power doing and redoing builds that will fail until
> something else has been built (which is the problem with mock's
> --chain command, if I understand correctly).
> 
> Once I have that graph, I use Make to control the process, because it
> handles the dependency graph, as well as parallelism, and not
> rebuilding things unnecessarily.

Yeah, all this ^ 

> > in it. However, there are time consuming challenges with this:
> > 
> > 1) False failures. Sometimes, the copr build fails for random reason
> > (Koji repo is not available, etc.). I need to read the logs and figure
> > it out, resubmit the build.
> > 
> > 2) Unrelated failures. Many packages FTBFS for unrelated reasons. I need
> > to spin up another (control) copr and rebuild all failures there as well
> > to make sure it's indeed unrelated.
> 
> Yes, that's a real pain. Can we just add everything to Koschei instead
> of having it opt-in?

Thats worth considering in the new year. I would like that. :) 

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[José Abílio Matos]

On Wednesday, December 16, 2020 7:15:21 PM WET Jonathan Wakely wrote:
> Creating the dependency graph by hand is fairly tedious, but maybe I'm
> missing an automated way. The point of creating that graph is to avoid
> wasting time and power doing and redoing builds that will fail until
> something else has been built (which is the problem with mock's
> --chain command, if I understand correctly).
> 
> Once I have that graph, I use Make to control the process, because it
> handles the dependency graph, as well as parallelism, and not
> rebuilding things unnecessarily.

If I am reading it right this is a task for "Fed Brunch":
https://github.com/juhp/fbrnch#readme 

Jens Petersen did a presentation about this at Fedora Nest this year, the 
package is available in copr:
https://copr.fedorainfracloud.org/coprs/petersen/fbrnch/ 

Honestly I have not yet tried it. I have been busy with other projects but 
this seems to fulfill some of your specifications.

Regards,
-- 
José Abílio___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Fabio Valentini]

On Wed, Dec 16, 2020 at 8:29 PM Adam Williamson
 wrote:
>

(snip)

> What I would prefer to do is get a reliable reverse dep check (we may
> have one already, I haven't looked) and gate updates on it. This would
> require soname bumps for Rawhide to be sent out as multi-package
> updates containing rebuilds of all dependent packages.
>
> This would be a big change, but I think we're at the point where we
> should really think about making it happen. We basically have all the
> necessary pieces to do it at this point, and the experience. We just
> need to piece through all the work and communication required. We might
> need to re-examine some permissions issues as part of it, too
> (including Bodhi's rather strict permission rules for editing updates).
>
> As a preview, after the RH shutdown - in the New Year - I'm considering
> seriously pushing some proposals for heavier gating across Fedora. I'm
> at least wanting to look at gating Rawhide composes (initially on a
> smaller set of required tests than the one we've been prototyping for a
> while), gating critpath updates on at least some of the openQA tests,
> and getting a subset of openQA update tests running on Rawhide critpath
> updates and considering if we can start doing some Rawhide update
> gating as per above.

Most people don't remember, but I proposed to FESCo to make the
dist.rpmdeplint gating check run by default, but it was decommissioned
with taskotron. IIRC, we decided to approve making any reverse dep
check a gating check if it were implemented on top of the new Fedora
CI again:
https://pagure.io/fesco/issue/2343

I'm not sure if the check was reimplemented for the new system yet,
but I seem to remember somebody talking about it lately. So if that
actually works, please propose to enable it by default, FESCo will
probably like it :)

Fabio
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Fabio Valentini]

On Wed, Dec 16, 2020 at 8:37 PM Matthew Miller  wrote:
>
> On Wed, Dec 16, 2020 at 09:11:53AM -0600, Richard Shaw wrote:
> > In the same vein of waste, but of resources rather than time, when I'm
> > updating a package I like to do a local mock build to check that upstream
> > didn't break anything and that if there are patches that they apply
> > cleanly.
>
> Koschei, basically?

I think you mean anitya / release-monitoring? It can be set up to do
koji scratch builds for new versions automatically. I think there were
plans to also create PRs automatically instead of attaching a patch to
the created BZ bug.

Fabio
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Florian Weimer]

* Adam Williamson:

> See I thought that too at first, and was going to cite it, but then I
> thought, wait. The problem isn't that the update *actually broke the
> ABI*, right? The problem is that it *unnecessarily bumped the soname*.
> I think abidiff's job is to catch the *opposite* problem, isn't it?
> Where the ABI changes but the soname isn't bumped.

Eh, the soname is part of the ABI provided by the package.  Historically
symbol versions were even tied to sonames (but glibc 2.30 did away with
that because all it did was preventing valid programs from loading).

Maybe the ABI checker employed here doesn't verify it because it assumes
that part is checked as part of the RPM dependencies?

Thanks,
Florian
-- 
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Matthew Miller]

On Wed, Dec 16, 2020 at 09:11:53AM -0600, Richard Shaw wrote:
> In the same vein of waste, but of resources rather than time, when I'm
> updating a package I like to do a local mock build to check that upstream
> didn't break anything and that if there are patches that they apply
> cleanly.

Koschei, basically?

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Adam Williamson]

On Wed, 2020-12-16 at 11:18 -0800, Kevin Fenzi wrote:
> On Wed, Dec 16, 2020 at 11:06:45AM -0800, Adam Williamson wrote:
> > 
> > See I thought that too at first, and was going to cite it, but then I
> > thought, wait. The problem isn't that the update *actually broke the
> > ABI*, right? The problem is that it *unnecessarily bumped the soname*.
> > I think abidiff's job is to catch the *opposite* problem, isn't it?
> > Where the ABI changes but the soname isn't bumped.
> > 
> > I would need to check, but I suspect possibly in this case abidiff just
> > wouldn't do anything at all, because what it would seem to make sense
> > to do is run it only on pairs of shared libraries with identical
> > sonames from the two package builds. When the soname is bumped, it
> > wouldn't make sense to run abidiff, because you'd *expect* the ABI to
> > change in that case.
> 
> Ah, indeed. That could well be the case.
> 
> I wonder if we shouldn't setup a 'soname bump test', make it gating for
> everything and require waiving it. It would be a extra step and more
> hassle, but it would prevent unintended soname bumps from landing, it
> would make it a deliberate choice. 

What I would prefer to do is get a reliable reverse dep check (we may
have one already, I haven't looked) and gate updates on it. This would
require soname bumps for Rawhide to be sent out as multi-package
updates containing rebuilds of all dependent packages.

This would be a big change, but I think we're at the point where we
should really think about making it happen. We basically have all the
necessary pieces to do it at this point, and the experience. We just
need to piece through all the work and communication required. We might
need to re-examine some permissions issues as part of it, too
(including Bodhi's rather strict permission rules for editing updates).

As a preview, after the RH shutdown - in the New Year - I'm considering
seriously pushing some proposals for heavier gating across Fedora. I'm
at least wanting to look at gating Rawhide composes (initially on a
smaller set of required tests than the one we've been prototyping for a
while), gating critpath updates on at least some of the openQA tests,
and getting a subset of openQA update tests running on Rawhide critpath
updates and considering if we can start doing some Rawhide update
gating as per above.
-- 
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Kevin Fenzi]

On Wed, Dec 16, 2020 at 11:06:45AM -0800, Adam Williamson wrote:
> 
> See I thought that too at first, and was going to cite it, but then I
> thought, wait. The problem isn't that the update *actually broke the
> ABI*, right? The problem is that it *unnecessarily bumped the soname*.
> I think abidiff's job is to catch the *opposite* problem, isn't it?
> Where the ABI changes but the soname isn't bumped.
> 
> I would need to check, but I suspect possibly in this case abidiff just
> wouldn't do anything at all, because what it would seem to make sense
> to do is run it only on pairs of shared libraries with identical
> sonames from the two package builds. When the soname is bumped, it
> wouldn't make sense to run abidiff, because you'd *expect* the ABI to
> change in that case.

Ah, indeed. That could well be the case.

I wonder if we shouldn't setup a 'soname bump test', make it gating for
everything and require waiving it. It would be a extra step and more
hassle, but it would prevent unintended soname bumps from landing, it
would make it a deliberate choice. 

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Jonathan Wakely]

On 15/12/20 23:46 +0100, Miro Hrončok wrote:

On 12/15/20 11:29 PM, Miroslav Suchý wrote:

I am looking for challenges for upcoming year - what I and my team should 
enhance. I have some ideas, but I want to hear
yours.


Thanks for doing this!


What you - as Fedora packager - find most time consuming on packaging?


Coordination and communication ;)


Where you will welcome more simplicity or automation?


In testing an impact of a change. E.g. a simple "upgrade to newer 
version" change might be a problem if it breaks other packages. I 
usually spin up a copr repo and try to rebuild every dependent package 


^ This.

The individual steps of doing a single package are insignificant
compared to the days or WEEKS it takes for a systemwide change that
requires rebuilding hundreds of dependencies. I know not many of us
actually have this problem, but for those who do, it's very, very time
consuming.

Since we're apparently not allowed to use a side tag to do a test
rebuild for this kind of thing, I end up doing it locally on my own
machines. Copr is another option, but I don't think it would be any
quicker or simpler.

What I'd really like would be a "test mass rebuild" process, where a
prospective package is uploaded and everything that depends on it is
automatically rebuilt (ideally after creating the dependency graph so
each individual package doesn't get rebuilt until its dependencies are
finished building).

Creating the dependency graph by hand is fairly tedious, but maybe I'm
missing an automated way. The point of creating that graph is to avoid
wasting time and power doing and redoing builds that will fail until
something else has been built (which is the problem with mock's
--chain command, if I understand correctly).

Once I have that graph, I use Make to control the process, because it
handles the dependency graph, as well as parallelism, and not
rebuilding things unnecessarily.


in it. However, there are time consuming challenges with this:

1) False failures. Sometimes, the copr build fails for random reason 
(Koji repo is not available, etc.). I need to read the logs and figure 
it out, resubmit the build.


2) Unrelated failures. Many packages FTBFS for unrelated reasons. I 
need to spin up another (control) copr and rebuild all failures there 
as well to make sure it's indeed unrelated.


Yes, that's a real pain. Can we just add everything to Koschei instead
of having it opt-in?

3) Cross-related failures. Some packages only fail in the test copr 
because they "see" other packages from that test copr and some of them 
are different from what Fedora rawhide offers (e.g. because git HEAD 
is newer than what has been built in Koji / passed CI in github). I 
sometimes need to create more isolated coprs with my change to rule 
this out.


Ideally, I'd like to see a CI job for this that handles this on the 
background. If you'd like to hear more about the manual workflow, I'd 
be happy to meet over video.


(The workflow is heavily based on https://github.com/fedora-stewardship/fedora-stewardship.github.io/blob/master/scripts/review_pr.py 
but recently, I prefer the new Copr's dist-git option.)

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Vitaly Zaitsev via devel]

On 16.12.2020 19:01, Adam Williamson wrote:

where we can see the "/abidiff" results:


Great. Thanks.

--
Sincerely,
  Vitaly Zaitsev (vit...@easycoding.org)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Adam Williamson]

On Wed, 2020-12-16 at 10:48 -0800, Kevin Fenzi wrote:
> On Wed, Dec 16, 2020 at 10:01:19AM -0800, Adam Williamson wrote:
> > On Wed, 2020-12-16 at 12:37 +0100, Vitaly Zaitsev via devel wrote:
> > > On 15.12.2020 23:29, Miroslav Suchý wrote:
> > > > What you - as Fedora packager - find most time consuming on packaging?
> > > 
> > > ABI check between shared libraries updates.
> > > 
> > > Bodhi used to run abidiff automated tests, but after Fedora Infra moved 
> > > to the new datacenter, they disappeared.
> > 
> > Bodhi doesn't run any tests, it only shows results from other systems.
> > 
> > abi check was run by Taskotron, which has been retired. The approximate
> > replacement is Fedora CI, which should run rpminspect (among other
> > tests). rpminspect runs abidiff. So there should still be abidiff
> > results, if Fedora CI is doing what it says it should.
> > 
> > For instance, here is the most recent Rawhide update from you:
> > https://bodhi.fedoraproject.org/updates/FEDORA-2020-af9f093288
> > 
> > If we click on "Automated Tests", we see a result for
> > "fedora-ci.koji-build.rpminspect.static-analysis". Clicking on it takes
> > us to:
> > https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pipeline/job/master/2629/testReport/(root)/tests/
> > 
> > where we can see the "/abidiff" results:
> > https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pipeline/job/master/2629/testReport/(root)/tests/_abidiff/
> > 
> > are OK. I don't know of a known failure case to check it's failing when
> > it should, though.
> 
> Well, rpm broke the buildroot eariler today so it should have failed
> this right?
> 
> https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pipeline/job/master/2840/testReport/(root)/tests/_abidiff/
> 
> but... no?
> 
> "Comparing rpm-4.16.1.1-1.fc34 with older rpm-4.16.1-1.fc34 found in the 
> "f34-updates" Koji tag."
> 
> it's not checking the actual previous build?

See I thought that too at first, and was going to cite it, but then I
thought, wait. The problem isn't that the update *actually broke the
ABI*, right? The problem is that it *unnecessarily bumped the soname*.
I think abidiff's job is to catch the *opposite* problem, isn't it?
Where the ABI changes but the soname isn't bumped.

I would need to check, but I suspect possibly in this case abidiff just
wouldn't do anything at all, because what it would seem to make sense
to do is run it only on pairs of shared libraries with identical
sonames from the two package builds. When the soname is bumped, it
wouldn't make sense to run abidiff, because you'd *expect* the ABI to
change in that case.
-- 
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Kevin Fenzi]

On Wed, Dec 16, 2020 at 10:01:19AM -0800, Adam Williamson wrote:
> On Wed, 2020-12-16 at 12:37 +0100, Vitaly Zaitsev via devel wrote:
> > On 15.12.2020 23:29, Miroslav Suchý wrote:
> > > What you - as Fedora packager - find most time consuming on packaging?
> > 
> > ABI check between shared libraries updates.
> > 
> > Bodhi used to run abidiff automated tests, but after Fedora Infra moved 
> > to the new datacenter, they disappeared.
> 
> Bodhi doesn't run any tests, it only shows results from other systems.
> 
> abi check was run by Taskotron, which has been retired. The approximate
> replacement is Fedora CI, which should run rpminspect (among other
> tests). rpminspect runs abidiff. So there should still be abidiff
> results, if Fedora CI is doing what it says it should.
> 
> For instance, here is the most recent Rawhide update from you:
> https://bodhi.fedoraproject.org/updates/FEDORA-2020-af9f093288
> 
> If we click on "Automated Tests", we see a result for
> "fedora-ci.koji-build.rpminspect.static-analysis". Clicking on it takes
> us to:
> https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pipeline/job/master/2629/testReport/(root)/tests/
> 
> where we can see the "/abidiff" results:
> https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pipeline/job/master/2629/testReport/(root)/tests/_abidiff/
> 
> are OK. I don't know of a known failure case to check it's failing when
> it should, though.

Well, rpm broke the buildroot eariler today so it should have failed
this right?

https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pipeline/job/master/2840/testReport/(root)/tests/_abidiff/

but... no?

"Comparing rpm-4.16.1.1-1.fc34 with older rpm-4.16.1-1.fc34 found in the 
"f34-updates" Koji tag."

it's not checking the actual previous build?

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Adam Williamson]

On Wed, 2020-12-16 at 12:37 +0100, Vitaly Zaitsev via devel wrote:
> On 15.12.2020 23:29, Miroslav Suchý wrote:
> > What you - as Fedora packager - find most time consuming on packaging?
> 
> ABI check between shared libraries updates.
> 
> Bodhi used to run abidiff automated tests, but after Fedora Infra moved 
> to the new datacenter, they disappeared.

Bodhi doesn't run any tests, it only shows results from other systems.

abi check was run by Taskotron, which has been retired. The approximate
replacement is Fedora CI, which should run rpminspect (among other
tests). rpminspect runs abidiff. So there should still be abidiff
results, if Fedora CI is doing what it says it should.

For instance, here is the most recent Rawhide update from you:
https://bodhi.fedoraproject.org/updates/FEDORA-2020-af9f093288

If we click on "Automated Tests", we see a result for
"fedora-ci.koji-build.rpminspect.static-analysis". Clicking on it takes
us to:
https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pipeline/job/master/2629/testReport/(root)/tests/

where we can see the "/abidiff" results:
https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pipeline/job/master/2629/testReport/(root)/tests/_abidiff/

are OK. I don't know of a known failure case to check it's failing when
it should, though.
-- 
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Robbie Harwood]

Miroslav Suchý  writes:

> I am looking for challenges for upcoming year - what I and my team
> should enhance. I have some ideas, but I want to hear yours.
>
> What you - as Fedora packager - find most time consuming on packaging?
> Where you will welcome more simplicity or automation?

Babysitting the underprovisioned architectures (s390x is the biggest
offender here), especially when they don't have test machines (
https://fedoraproject.org/wiki/Test_Machine_Resources_For_Package_Maintainers
).

Thanks,
--Robbie


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Richard Shaw]

On Wed, Dec 16, 2020 at 8:56 AM Artur Frenszek-Iwicki <
s...@fedoraproject.org> wrote:

> Less of "very time consuming" and more of "occasional annoyance": "fedpkg
> commit" could inspect the .spec file and ensure that every Source has a
> matching entry in the sources file, and that every Patch has a
> corresponding file in the repo. Every now and then I do a local build, it
> works, I commit forgetting to run "fedpkg new-sources" and then the koji
> build fails.
>
> I know that I could avoid this by using "fedpkg mockbuild" instead of
> "fedpkg local", but still, I believe that adding extra checks to reduce the
> possibility of human error is a good thing.
>

In the same vein of waste, but of resources rather than time, when I'm
updating a package I like to do a local mock build to check that upstream
didn't break anything and that if there are patches that they apply
cleanly.

If I haven't done a "fedpkg new-source ..." then it blindly downloads the
current source from the sources file even though it is no longer referenced
in the spec file. It could check to see if the sources in the spec file
match what's in sources. So I end up uploading the new sources (without
commiting) to work around this behavior. But what if there's a problem and
I end up choosing not to build the package? The file is already uploaded...

For small packages it's not a huge deal, just an annoyance, but there are
packages that the upstream sources are more than 300MB.

Thanks,
Richard
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Artur Frenszek-Iwicki]

Less of "very time consuming" and more of "occasional annoyance": "fedpkg 
commit" could inspect the .spec file and ensure that every Source has a 
matching entry in the sources file, and that every Patch has a corresponding 
file in the repo. Every now and then I do a local build, it works, I commit 
forgetting to run "fedpkg new-sources" and then the koji build fails. 

I know that I could avoid this by using "fedpkg mockbuild" instead of "fedpkg 
local", but still, I believe that adding extra checks to reduce the possibility 
of human error is a good thing.

Alternatively, this could be done server-side in a hook, rejecting any pushes 
that have missing files.

A.FI.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Fabio Valentini]

On Wed, Dec 16, 2020 at 2:12 PM Artem Tim  wrote:
>
> Rust packaging automation. People shouldn't do work which could be done by 
> machine in 2020.

rust2rpm is already pretty good, but I have some ideas to make the
possibilities for automation more comprehensive. I still need to write
the code, though.

Fabio
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Petr Šabata]

On Tue, Dec 15, 2020 at 11:33 PM Miroslav Suchý  wrote:
>
> I am looking for challenges for upcoming year - what I and my team should 
> enhance. I have some ideas, but I want to hear
> yours.
>
> What you - as Fedora packager - find most time consuming on packaging?
> Where you will welcome more simplicity or automation?

I suppose many of us have scripts or aliases for this kind of thing
but maybe something could be integrated into the tools to make it
easier:

Review changes in the upstream sources; maybe just a diff between the
two tarballs would be sufficient. This helps one identify changes in
dependencies, potential breaking behavior, new files or features
(hello, shell completions), minor changes in licensing, and so on.

Not every upstream has a reasonable SCM one can inspect. And not every
upstream provides a changelog, or a changelog that can be trusted.

P
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Artem Tim]

Rust packaging automation. People shouldn't do work which could be done by 
machine in 2020.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Tim Landscheidt]

Josh Boyer  wrote:

>  I am looking for challenges for upcoming year - what I and my team should 
> enhance. I have some ideas, but I want to hear
>  yours.

>  What you - as Fedora packager - find most time consuming on packaging?
>  Where you will welcome more simplicity or automation?

> Deciphering whatever packaging guidelines have changed since the last time I 
> looked and trying to figure out how to update packages to adhere to them even 
> though nobody audits the package set
> anyway.

> […]

I really like Debian's "Standards-Version" field
(https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-standards-version)
in that context; it does not tell what changes in the guide-
lines have occured, but it makes it unambiguous what guide-
lines the package /should/ comply with.

Tim
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Peter Hanecak]

Hello,

from top-of-the-head: I'm always referring to basic packaging manual and 
main "blocker" is waiting for builds, so that I can push updates 
afterwards: either I wait till completion (=waste of time on my end) or 
interrupt the package update session (=users will get updates later).


Speeding up builds might help, but maybe adjustments in workflow 
(avoiding update push being blocked by build) might be even better. But 
I'm not sure whether that's possible or feasible.


Sincerely

Peter


On 12/15/20 11:29 PM, Miroslav Suchý wrote:

I am looking for challenges for upcoming year - what I and my team should 
enhance. I have some ideas, but I want to hear
yours.

What you - as Fedora packager - find most time consuming on packaging?
Where you will welcome more simplicity or automation?



--
Peter Hanecak
  http://hany.sk/~hany/
  GnuPG: http://hany.sk/~hany/gpg/475DFC4C.txt
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Vitaly Zaitsev via devel]

On 15.12.2020 23:29, Miroslav Suchý wrote:

What you - as Fedora packager - find most time consuming on packaging?


ABI check between shared libraries updates.

Bodhi used to run abidiff automated tests, but after Fedora Infra moved 
to the new datacenter, they disappeared.


--
Sincerely,
  Vitaly Zaitsev (vit...@easycoding.org)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Adam Williamson]

On Tue, 2020-12-15 at 18:29 -0500, Matthew Miller wrote:
> On Tue, Dec 15, 2020 at 04:44:49PM -0600, Michael Cronenworth wrote:
> > Can we make the Bodhi update notes part of the git repo? Ex. ChangeLog.md
> 
> I would definitely like to see a consolidated changelog. I don't know if it
> should be part of the git repo, or just a changelog convention. (Like:
> any lines starting with > get put into the notes for the next bodhi update?)

My traditional note that anything along these lines should be optional.
I frequently submit multi-package updates, and want to write the update
text directly, separate from package commit messages and changelogs.
For me, the description of "this update containing five new package
builds" is completely different from the package-level description of
any one of those new builds.
-- 
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Michael Catanzaro]

On Tue, Dec 15, 2020 at 6:29 pm, Matthew Miller 
 wrote:
I would definitely like to see a consolidated changelog. I don't know 
if it

should be part of the git repo, or just a changelog convention. (Like:
any lines starting with > get put into the notes for the next bodhi 
update?)


+1 to consolidated changelog.

Changelog should really not be part of the spec file anymore.

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Matthew Miller]

On Tue, Dec 15, 2020 at 04:44:49PM -0600, Michael Cronenworth wrote:
> Can we make the Bodhi update notes part of the git repo? Ex. ChangeLog.md

I would definitely like to see a consolidated changelog. I don't know if it
should be part of the git repo, or just a changelog convention. (Like:
any lines starting with > get put into the notes for the next bodhi update?)

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Josh Boyer]

On Tue, Dec 15, 2020, 5:29 PM Miroslav Suchý  wrote:

> I am looking for challenges for upcoming year - what I and my team should
> enhance. I have some ideas, but I want to hear
> yours.
>
> What you - as Fedora packager - find most time consuming on packaging?
> Where you will welcome more simplicity or automation?
>

Deciphering whatever packaging guidelines have changed since the last time
I looked and trying to figure out how to update packages to adhere to them
even though nobody audits the package set anyway.

Followed closely by taking whatever the upstream tracking service does and
manually applying it and rebuilding it.  It would be so much better if it
just filed a PR and kicked off a scratch build as part of the ci.

josh
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Miro Hrončok]

On 12/15/20 11:29 PM, Miroslav Suchý wrote:

I am looking for challenges for upcoming year - what I and my team should 
enhance. I have some ideas, but I want to hear
yours.


Thanks for doing this!


What you - as Fedora packager - find most time consuming on packaging?


Coordination and communication ;)


Where you will welcome more simplicity or automation?


In testing an impact of a change. E.g. a simple "upgrade to newer version" 
change might be a problem if it breaks other packages. I usually spin up a copr 
repo and try to rebuild every dependent package in it. However, there are time 
consuming challenges with this:


1) False failures. Sometimes, the copr build fails for random reason (Koji repo 
is not available, etc.). I need to read the logs and figure it out, resubmit the 
build.


2) Unrelated failures. Many packages FTBFS for unrelated reasons. I need to spin 
up another (control) copr and rebuild all failures there as well to make sure 
it's indeed unrelated.


3) Cross-related failures. Some packages only fail in the test copr because they 
"see" other packages from that test copr and some of them are different from 
what Fedora rawhide offers (e.g. because git HEAD is newer than what has been 
built in Koji / passed CI in github). I sometimes need to create more isolated 
coprs with my change to rule this out.


Ideally, I'd like to see a CI job for this that handles this on the background. 
If you'd like to hear more about the manual workflow, I'd be happy to meet over 
video.


(The workflow is heavily based on 
https://github.com/fedora-stewardship/fedora-stewardship.github.io/blob/master/scripts/review_pr.py 
but recently, I prefer the new Copr's dist-git option.)


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: What is the most time consuming task for you as packager?

[Michael Cronenworth]

On 12/15/20 4:29 PM, Miroslav Suchý wrote:

What you - as Fedora packager - find most time consuming on packaging?
Where you will welcome more simplicity or automation?


Pushing updates requires too many steps, IMHO.

1. 
2. fedpkg mockbuild
3. 
4. fedpkg commit -p
5. fedpkg build
6. fedpkg update ...

Can we make the Bodhi update notes part of the git repo? Ex. ChangeLog.md
If not, could a file be referenced instead of a "--notes" argument? Ex. --notes-file 
ChangeLog.md

Stretch goal: Bodhi update defaults (inc. notes?) as part of git, too. Ex. 
update.yml

Step 4 could then become the final step:
fedpkg commit -p -b -u
(-b = build, -u = send update if build is successful)

Regards,
Michael
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org