[minimization] Feedback Pipeline feedback wanted

[Adam Samalik]

The Minimization Objective[1] has been going on for a while. There are two
high-level goals: making things smaller, and keeping things smaller. On the
keeping smaller side, the team prototyped a service called Feedback
Pipeline [2] that monitors use cases for their installation size and
dependencies, including a size history. This will help us see bigger
changes in size for things the community cares about [3].

I already got some feedback from a few individuals I asked while developing
it, but I feel it's in a good enough state for a more broad feedback. So I
have a few questions:

1/ We plan to send weekly size updates to the devel list. Would that be
useful? What should they include?

2/ Regarding the use cases [4], especially the container ones, could people
please review and give feedback to those? Are all the packages there
actually required? I'm specifically looking at the "nss_wrapper" package
that drags in Perl and cmake which makes it huge.

3/ Are there any other use cases we should track? I'm sure there are!

4/ And a more general one: is there something you're working on that's
related Minimization? Please let me know.

Cheers,
Adam

PS: The service is a prototype, so please excuse if there are some rough
edges — like the history graph showing multiple values of the same color.
If you happen to be interested to contribute, I'd be glad to accept issues
(or even PRs!) in the repo [5].


[1] Objective: https://docs.fedoraproject.org/en-US/minimization/
[2] Feedback Pipeline: https://minimization.github.io
\
[3] Things the community cares about: There is an initial list [4] of use
cases defined by the Minimization Team, and we're now looking for feedback
and suggestions for new ones.
[4] Use cases:
https://minimization.github.io/reports/view--use-cases-definitions.html
[5] Feedback Pipeline repo:
https://github.com/minimization/feedback-pipeline/

-- 

Adam Šamalík
---
Senior Software Engineer
Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [minimization] Feedback Pipeline feedback wanted

[Vít Ondruch]

Dne 12. 12. 19 v 12:59 Adam Samalik napsal(a):
> The Minimization Objective[1] has been going on for a while. There are
> two high-level goals: making things smaller, and keeping things
> smaller. On the keeping smaller side, the team prototyped a service
> called Feedback Pipeline [2] that monitors use cases for their
> installation size and dependencies, including a size history. This
> will help us see bigger changes in size for things the community cares
> about [3].
>
> I already got some feedback from a few individuals I asked while
> developing it, but I feel it's in a good enough state for a more broad
> feedback. So I have a few questions:
>
> 1/ We plan to send weekly size updates to the devel list. Would that
> be useful? What should they include?


The compose report already includes size changes. So if there was some
statistic included, that would be nice. Also if there was list of top
increases and decreases, that would be probably useful.


Vít



>
> 2/ Regarding the use cases [4], especially the container ones, could
> people please review and give feedback to those? Are all the packages
> there actually required? I'm specifically looking at the "nss_wrapper"
> package that drags in Perl and cmake which makes it huge.
>
> 3/ Are there any other use cases we should track? I'm sure there are!
>
> 4/ And a more general one: is there something you're working on that's
> related Minimization? Please let me know.
>
> Cheers,
> Adam
>
> PS: The service is a prototype, so please excuse if there are some
> rough edges — like the history graph showing multiple values of the
> same color. If you happen to be interested to contribute, I'd be glad
> to accept issues (or even PRs!) in the repo [5]. 
>
>
> [1] Objective: https://docs.fedoraproject.org/en-US/minimization/
> [2] Feedback Pipeline: https://minimization.github.io
> \
> [3] Things the community cares about: There is an initial list [4] of
> use cases defined by the Minimization Team, and we're now looking for
> feedback and suggestions for new ones.
> [4] Use
> cases: https://minimization.github.io/reports/view--use-cases-definitions.html
> [5] Feedback Pipeline
> repo: https://github.com/minimization/feedback-pipeline/
>
> -- 
>
> Adam Šamalík
> ---
> Senior Software Engineer
> Red Hat
>
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [minimization] Feedback Pipeline feedback wanted

[Kevin Fenzi]

On Thu, Dec 12, 2019 at 12:59:03PM +0100, Adam Samalik wrote:
> The Minimization Objective[1] has been going on for a while. There are two
> high-level goals: making things smaller, and keeping things smaller. On the
> keeping smaller side, the team prototyped a service called Feedback
> Pipeline [2] that monitors use cases for their installation size and
> dependencies, including a size history. This will help us see bigger
> changes in size for things the community cares about [3].
> 
> I already got some feedback from a few individuals I asked while developing
> it, but I feel it's in a good enough state for a more broad feedback. So I
> have a few questions:
> 
> 1/ We plan to send weekly size updates to the devel list. Would that be
> useful? What should they include?

It would be great if they could include the size +/- of all the images. 
Of course the most important ones would be boot.iso, workstation and
server, but labs and spins could be very helpfull as well. 
> 
> 2/ Regarding the use cases [4], especially the container ones, could people
> please review and give feedback to those? Are all the packages there
> actually required? I'm specifically looking at the "nss_wrapper" package
> that drags in Perl and cmake which makes it huge.

Which use case is that? All of the container ones?
Sounds like a nice link to drop, but someone would need to find out the
details. Why is it included?

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [minimization] Feedback Pipeline feedback wanted

[Adam Williamson]

On Thu, 2019-12-12 at 12:59 +0100, Adam Samalik wrote:
> The Minimization Objective[1] has been going on for a while. There are two
> high-level goals: making things smaller, and keeping things smaller. On the
> keeping smaller side, the team prototyped a service called Feedback
> Pipeline [2] that monitors use cases for their installation size and
> dependencies, including a size history. This will help us see bigger
> changes in size for things the community cares about [3].

Just a note: I've actually already been doing something similar for a
while. The openQA tests record various information on an installed
system after install completes, and check-compose performs some
analysis on this information. Significant changes are included in the
'compose check report' emails sent to this list and test@ . For
instance, a notable change in the size of the installed system is
reported, as are changes to the installed package set, or to the set of
default-enabled system services.

https://pagure.io/fedora-qa/os-autoinst-distri-fedora/blob/master/f/tests/_collect_data.pm
https://pagure.io/fedora-qa/check-compose/blob/master/f/check-compose#_306

I've always said I'm willing to enhance this in various ways (e.g. by
having reporting in other ways than an email, like a web service), but
no-one had really expressed any interest so far, so I haven't
prioritized it.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [minimization] Feedback Pipeline feedback wanted

[Adam Williamson]

On Fri, 2019-12-13 at 09:19 -0800, Kevin Fenzi wrote:
> It would be great if they could include the size +/- of all the images. 
> Of course the most important ones would be boot.iso, workstation and
> server, but labs and spins could be very helpfull as well.

This sort of thing is what fedfind can help with. As an example of
where you'd start:

#!/bin/python3

import fedfind.release
import fedfind.helpers

rel = fedfind.release.get_release(cid="Fedora-Rawhide-20191213.n.0")
#rel = fedfind.release.get_release(1)
for img in rel.all_images:
imgid = fedfind.helpers.identify_image(img, out='string')
if img['disc_number'] > 1:
imgid += " disc {0}".format(img['disc_number'])
size = img.get('size')
if not size:
# this is something I should make fedfind handle...
# I swear it used to!
size = fedfind.helpers.get_size(img['direct_url'])
print("{0}: {1}".format(imgid, size))

hopefully it's pretty simple to see where to go from there. :) You can
try it with either of the 'rel' lines and it'll work...so you can
compare the image sizes from Fedora Core 1 with those from today's
Rawhide nightly, though they only have one image entirely in common
(Everything-boot-iso).

You can see I had to add a couple of bits to smoothly work with the
info for very old composes...I'll maybe tweak that a bit in fedfind
itself today, that code doesn't actually get *used* a lot so when I do
want to do something like this I usually find some issues that have
crept in.

This should work for any Pungi 4 compose that hasn't been garbage
collected yet, and also for all stable releases and old candidate
composes.

It...*could* also work for non-candidate (i.e. nightly) Pungi 4
composes that have been garbage collected by retrieving the info from
PDC, but this thread has made me realize that's a path that I've sort
of shut off with some design choices and I need to think about how to
open it up again.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [minimization] Feedback Pipeline feedback wanted

[Adam Williamson]

On Fri, 2019-12-13 at 11:17 -0800, Adam Williamson wrote:
> 
> It...*could* also work for non-candidate (i.e. nightly) Pungi 4
> composes that have been garbage collected by retrieving the info from
> PDC, but this thread has made me realize that's a path that I've sort
> of shut off with some design choices and I need to think about how to
> open it up again.

So count me as thoroughly nerdsniped - I spent all of today coming up
with a (somewhat icky) fix for this:

https://pagure.io/fedora-qa/fedfind/c/3e7a261be9faf9045ac2a5b3ba648536af86b010?branch=master

so with fedfind git master, the sample script now will also give you
date for old composes that are no longer present on any mirror, but for
which the metadata was uploaded to PDC. Try it with e.g.
Fedora-Rawhide-20171230.n.1 .

I also thought about the 'get image sizes for old releases that are
still on the mirrors' problem a bit and came up with this:

https://pagure.io/quick-fedora-mirror/pull-request/82

which basically enhances the magic file fedfind uses to *find* all the
image files from old releases to include each file's size as well; if
tibbs is OK with that change, I can enhance fedfind to use that info
and include the size in the image dicts it synthesizes for old
releases.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [minimization] Feedback Pipeline feedback wanted

[Adam Samalik]

On Fri, Dec 13, 2019 at 8:20 PM Adam Williamson 
wrote:

> On Fri, 2019-12-13 at 09:19 -0800, Kevin Fenzi wrote:
> > It would be great if they could include the size +/- of all the images.
> > Of course the most important ones would be boot.iso, workstation and
> > server, but labs and spins could be very helpfull as well.
>
> This sort of thing is what fedfind can help with. As an example of
> where you'd start:
>
> #!/bin/python3
>
> import fedfind.release
> import fedfind.helpers
>
> rel = fedfind.release.get_release(cid="Fedora-Rawhide-20191213.n.0")
> #rel = fedfind.release.get_release(1)
> for img in rel.all_images:
> imgid = fedfind.helpers.identify_image(img, out='string')
> if img['disc_number'] > 1:
> imgid += " disc {0}".format(img['disc_number'])
> size = img.get('size')
> if not size:
> # this is something I should make fedfind handle...
> # I swear it used to!
> size = fedfind.helpers.get_size(img['direct_url'])
> print("{0}: {1}".format(imgid, size))
>
> hopefully it's pretty simple to see where to go from there. :) You can
> try it with either of the 'rel' lines and it'll work...so you can
> compare the image sizes from Fedora Core 1 with those from today's
> Rawhide nightly, though they only have one image entirely in common
> (Everything-boot-iso).
>
> You can see I had to add a couple of bits to smoothly work with the
> info for very old composes...I'll maybe tweak that a bit in fedfind
> itself today, that code doesn't actually get *used* a lot so when I do
> want to do something like this I usually find some issues that have
> crept in.
>
> This should work for any Pungi 4 compose that hasn't been garbage
> collected yet, and also for all stable releases and old candidate
> composes.
>
> It...*could* also work for non-candidate (i.e. nightly) Pungi 4
> composes that have been garbage collected by retrieving the info from
> PDC, but this thread has made me realize that's a path that I've sort
> of shut off with some design choices and I need to think about how to
> open it up again.
>

That is really cool. And I'm aware of the compose report that goes to devel.

What I'm looking for is sizes of (and other info about) some specific use
cases,
that we don't actually produce as artifacts. Like the httpd web server
installed
on top of an image we produce, etc. They represent something that people
often install themselves as opposed us producing an image.

My goal is to detect changes that affect those use cases. It's conceptually
similar to some parts of the compose report, just looking at a different
thing.

There is also one technical difference: Because we don't produce the use
cases
as artifacts (we produce the very useful bits people can use to build
those),
the service also needs to build them so it actually has something to look
at.

So I prototyped the service to validate its usefulness. And I'd be happy to
throw away most of the code if something similar exists already — which was
the reason it's not really optimized.

Now I'm thinking if I could somehow use fedfind or the CI infrastructure
for that...



> --
> Adam Williamson
> Fedora QA Community Monkey
> IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
> http://www.happyassassin.net
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>


-- 

Adam Šamalík
---
Senior Software Engineer
Red Hat
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: [minimization] Feedback Pipeline feedback wanted

[Adam Williamson]

On Thu, 2019-12-19 at 13:10 +0100, Adam Samalik wrote:
> On Fri, Dec 13, 2019 at 8:20 PM Adam Williamson 
> wrote:
> 
> > On Fri, 2019-12-13 at 09:19 -0800, Kevin Fenzi wrote:
> > > It would be great if they could include the size +/- of all the images.
> > > Of course the most important ones would be boot.iso, workstation and
> > > server, but labs and spins could be very helpfull as well.
> > 
> > This sort of thing is what fedfind can help with. As an example of
> > where you'd start:
> > 
> > #!/bin/python3
> > 
> > import fedfind.release
> > import fedfind.helpers
> > 
> > rel = fedfind.release.get_release(cid="Fedora-Rawhide-20191213.n.0")
> > #rel = fedfind.release.get_release(1)
> > for img in rel.all_images:
> > imgid = fedfind.helpers.identify_image(img, out='string')
> > if img['disc_number'] > 1:
> > imgid += " disc {0}".format(img['disc_number'])
> > size = img.get('size')
> > if not size:
> > # this is something I should make fedfind handle...
> > # I swear it used to!
> > size = fedfind.helpers.get_size(img['direct_url'])
> > print("{0}: {1}".format(imgid, size))
> > 
> > hopefully it's pretty simple to see where to go from there. :) You can
> > try it with either of the 'rel' lines and it'll work...so you can
> > compare the image sizes from Fedora Core 1 with those from today's
> > Rawhide nightly, though they only have one image entirely in common
> > (Everything-boot-iso).
> > 
> > You can see I had to add a couple of bits to smoothly work with the
> > info for very old composes...I'll maybe tweak that a bit in fedfind
> > itself today, that code doesn't actually get *used* a lot so when I do
> > want to do something like this I usually find some issues that have
> > crept in.
> > 
> > This should work for any Pungi 4 compose that hasn't been garbage
> > collected yet, and also for all stable releases and old candidate
> > composes.
> > 
> > It...*could* also work for non-candidate (i.e. nightly) Pungi 4
> > composes that have been garbage collected by retrieving the info from
> > PDC, but this thread has made me realize that's a path that I've sort
> > of shut off with some design choices and I need to think about how to
> > open it up again.
> > 
> 
> That is really cool. And I'm aware of the compose report that goes to devel.
> 
> What I'm looking for is sizes of (and other info about) some specific use
> cases,
> that we don't actually produce as artifacts. Like the httpd web server
> installed
> on top of an image we produce, etc. They represent something that people
> often install themselves as opposed us producing an image.

Right, I was replying to Kevin's suggestion, which *is* about the sizes
of the images. I guess we can say that since fedfind can do the above,
either the Feedback Pipeline wouldn't need to implement Kevin's
suggestion, or if it wants to be the frontend for it, fedfind could
potentially help on the back end.

> My goal is to detect changes that affect those use cases. It's conceptually
> similar to some parts of the compose report, just looking at a different
> thing.
> 
> There is also one technical difference: Because we don't produce the use
> cases
> as artifacts (we produce the very useful bits people can use to build
> those),
> the service also needs to build them so it actually has something to look
> at.
> 
> So I prototyped the service to validate its usefulness. And I'd be happy to
> throw away most of the code if something similar exists already — which was
> the reason it's not really optimized.
> 
> Now I'm thinking if I could somehow use fedfind or the CI infrastructure
> for that...

Yeah, going back to the overall idea of the Feedback Pipeline, I just
wanted to make sure you were aware of this so we can avoid any
duplication if possible. If it turns out that fedfind/check-compose
don't help, hey, never mind. I'm happy to look at extending it in ways
that make sense if it could be useful, though doing some things may be
architecturally difficult and not worth the effort - it really depends
a lot on the details. I'm happy to kick those details around with you
any time, IRC or email :)

One thing I am going to do if I can find time is somehow set things up
so the data files we collect from openQA tests that check-compose uses
as its inputs don't get garbage collected after a while. We *should*
have interesting data on service loadouts and so on going back to the
date when I first started having openQA collect this data, but because
the files get garbage collected, we don't :( I will try and fix that
for sure, so if nothing else, systems like this can use that historical
data going forward.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject

Re: [minimization] Feedback Pipeline feedback wanted

[Adam Williamson]

On Fri, 2019-12-13 at 17:13 -0800, Adam Williamson wrote:
> On Fri, 2019-12-13 at 11:17 -0800, Adam Williamson wrote:
> > It...*could* also work for non-candidate (i.e. nightly) Pungi 4
> > composes that have been garbage collected by retrieving the info from
> > PDC, but this thread has made me realize that's a path that I've sort
> > of shut off with some design choices and I need to think about how to
> > open it up again.
> 
> So count me as thoroughly nerdsniped - I spent all of today coming up
> with a (somewhat icky) fix for this:
> 
> https://pagure.io/fedora-qa/fedfind/c/3e7a261be9faf9045ac2a5b3ba648536af86b010?branch=master
> 
> so with fedfind git master, the sample script now will also give you
> date for old composes that are no longer present on any mirror, but for
> which the metadata was uploaded to PDC. Try it with e.g.
> Fedora-Rawhide-20171230.n.1 .
> 
> I also thought about the 'get image sizes for old releases that are
> still on the mirrors' problem a bit and came up with this:
> 
> https://pagure.io/quick-fedora-mirror/pull-request/82
> 
> which basically enhances the magic file fedfind uses to *find* all the
> image files from old releases to include each file's size as well; if
> tibbs is OK with that change, I can enhance fedfind to use that info
> and include the size in the image dicts it synthesizes for old
> releases.

Just to follow up on this a bit more: I'm currently sending a new
fedfind release, 4.3.0, out to Bodhi with both these changes. Once it
goes stable for all releases I'll ask tibbs to merge the quick-fedora-
mirror PR so the real imagelist files get the size data, and then sizes
for all old stable releases will be available via fedfind.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org