Re: Remove gcc, gcc-c++ and make from minimal build root

[Dan Horák]

sorry for top-post, but one more thing to consider, if caching would be
used, are secondary arches - how it would affect their ability to build
the buildroots


Dan

On Sat, 17 Jan 2015 11:07:53 -0600
Dennis Gilmore den...@ausil.us wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 On Sat, 17 Jan 2015 03:44:10 +
 Peter Robinson pbrobin...@gmail.com wrote:
 
  that all being said. koji doesn't use any caching and
  will not use the lvm plugin. we make every buildroot from
  scratch using a fully clean environment to help with
  ensuring reproducability.

 You can cache and still preserve reproducability. What I'm
 planning for Copr is to do (every week/month) for chroot in
 fedora-20-x86_64 fedora-21_86_64 ... ; do mock --init
 $chroot done
 take snapshot of that. I plan to do that on VM level.
 And when new task come, I will just restore from that
 snapshot. And mock will start with already populated cache.
 So I will have better caching and yet reproducability.
you really can't.  you would need to make a new cache any time
one of the packages in the minimal buildroot changes.
  
   That is why mock run 'yum update' on minimal buildroot before
   proceeding further. So if one package in buildroot changes, you
   will just download and install just that one package and all
   other comes from cache.
  
   That invalidates being able to reproduce the build root exactly.
   as you would need to know and reproduce the package update. This
   is something that has undergone a lot of thought and
   consideration and there is very very good reasons things are the
   way they are. as i showed elsewhere in this thread most repos are
   used only once or twice at most on any given host.
  
  The only way you could cache a base build root effectively is if you
  track the packages and NVRs and when one of them bump you regenerate
  the buildroot, likely pausing all builds until a new one is there.
  there's likely not that much churn in the packages contained in the
  root buildroot (you'd want to make sure you had everything for
  building .src.rpms too) and in some cases you could likely get days
  on the same image without having to rebuild it, other days you
  might have to do it a few times a time.
 
 As I showed earlier in the thread most repos are only ever used once
 or twice on a host. but we do not know if something in the minimal
 buildroot has changed. there is a lot of added complexity needed to
 attempt to use caching. which adds many more chances for something to
 go wrong. I honestly do not know if its quicker to build the minimal
 buildroot from scratch or to unpack the tarball. there is costs both
 ways. with the added complexity in tracking things to try and make
 caching worthwhile.  in the Fedora case we would need to have
 somewhere in the order of 6-12 cached buildroots on each builder
 depending on sidetags etc in use at different times.  but there is
 instances of koji where that number would be in the hundreds. we need
 to make sure that it works and scales not only for our use cases but
 also other users of koji.
 
  Whether the effort to implement out weights the effort remains to be
  seen. I think, given the low IOPs on the builds with an underlying
  COP image it might well give a reasonable reduction in building
  especially during mass rebuilds where the builders are very active
  and as it's in a side tag the new build root packages don't end up
  in the build root until tagged. Using fedmsg it's likely not hard
  either to monitor a package subset for changes and regen the image.
 
 There is a massive effort to figure out if the effort to implement any
 caching is worth while. If someone wanted to take the time to figure
 that out, making sure to gather the needs of different koji users to
 make sure that we did not make things worse for them, then by all
 means that person is welcome to do so and I will gladly answer any
 and all questions that they had. However I have no interest in going
 down that rabbit hole. we can not rely on koji users using fedmsg so
 that is something that would likely need to be ruled out of the design
 decision. fedmsg also does not guarantee delivery of messages and so
 can not be relied on for anything where we have to be 100% sure that
 things are done given a trigger.
 
 Dennis
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v2
 
 iQIcBAEBAgAGBQJUupbqAAoJEH7ltONmPFDR+iQP/ietNky5y2yhtbUZ/MMGP1er
 PDMyHz5MD9v78mstXXjx6WsBrOjc2a51WCufBV2drlPuwyTrstwJShbOhm1XfOld
 1HSI9etIkwsVxBFgRfh6HUxbLGZWSHDZviY34KocxLNuuaRLqc5RHjTTZmzQwKl+
 4EA4o6HJAYPsVz0Fu2/sJAy9cCclfCPy8Mcpstd1NIHOPhtNTpFiEQBl4zpXRqQs
 nlmlNh1MhZepv84k+EvhRZCJlmfn0yLBLlNJfNh6jtdy6zv+lNf2oClrHDR9o5aL
 nRCEJhRl/LX+BNf2mV5MULxAliXMME1qoYrm//BgsQk45t3XvVJ2fpPEVs8gcr49
 gZ0yoJ20fcSW/UJCRMhoI9dtp1MYQmF3h3LHfNRROv+Ga9Uumw9OmQyFqFJwbF2o
 doLOvDxtpEXWsiXcTUSqcl3CBGyT50AiAbRvv5yrQ5TU1Z8TplcUVUbJ/zifBoid
 

Re: Remove gcc, gcc-c++ and make from minimal build root

[Dennis Gilmore]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Sat, 17 Jan 2015 03:44:10 +
Peter Robinson pbrobin...@gmail.com wrote:

 that all being said. koji doesn't use any caching and will
 not use the lvm plugin. we make every buildroot from
 scratch using a fully clean environment to help with
 ensuring reproducability.
   
You can cache and still preserve reproducability. What I'm
planning for Copr is to do (every week/month) for chroot in
fedora-20-x86_64 fedora-21_86_64 ... ; do mock --init $chroot
  done
take snapshot of that. I plan to do that on VM level.
And when new task come, I will just restore from that
snapshot. And mock will start with already populated cache.
So I will have better caching and yet reproducability.
   you really can't.  you would need to make a new cache any time
   one of the packages in the minimal buildroot changes.
 
  That is why mock run 'yum update' on minimal buildroot before
  proceeding further. So if one package in buildroot changes, you
  will just download and install just that one package and all other
  comes from cache.
 
  That invalidates being able to reproduce the build root exactly. as
  you would need to know and reproduce the package update. This is
  something that has undergone a lot of thought and consideration and
  there is very very good reasons things are the way they are. as i
  showed elsewhere in this thread most repos are used only once or
  twice at most on any given host.
 
 The only way you could cache a base build root effectively is if you
 track the packages and NVRs and when one of them bump you regenerate
 the buildroot, likely pausing all builds until a new one is there.
 there's likely not that much churn in the packages contained in the
 root buildroot (you'd want to make sure you had everything for
 building .src.rpms too) and in some cases you could likely get days on
 the same image without having to rebuild it, other days you might have
 to do it a few times a time.

As I showed earlier in the thread most repos are only ever used once or
twice on a host. but we do not know if something in the minimal
buildroot has changed. there is a lot of added complexity needed to
attempt to use caching. which adds many more chances for something to
go wrong. I honestly do not know if its quicker to build the minimal
buildroot from scratch or to unpack the tarball. there is costs both
ways. with the added complexity in tracking things to try and make
caching worthwhile.  in the Fedora case we would need to have somewhere
in the order of 6-12 cached buildroots on each builder depending on
sidetags etc in use at different times.  but there is instances of koji
where that number would be in the hundreds. we need to make sure that
it works and scales not only for our use cases but also other users of
koji.

 Whether the effort to implement out weights the effort remains to be
 seen. I think, given the low IOPs on the builds with an underlying COP
 image it might well give a reasonable reduction in building especially
 during mass rebuilds where the builders are very active and as it's in
 a side tag the new build root packages don't end up in the build root
 until tagged. Using fedmsg it's likely not hard either to monitor a
 package subset for changes and regen the image.

There is a massive effort to figure out if the effort to implement any
caching is worth while. If someone wanted to take the time to figure
that out, making sure to gather the needs of different koji users to
make sure that we did not make things worse for them, then by all means
that person is welcome to do so and I will gladly answer any and all
questions that they had. However I have no interest in going down that
rabbit hole. we can not rely on koji users using fedmsg so that is
something that would likely need to be ruled out of the design
decision. fedmsg also does not guarantee delivery of messages and so
can not be relied on for anything where we have to be 100% sure that
things are done given a trigger.

Dennis
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBAgAGBQJUupbqAAoJEH7ltONmPFDR+iQP/ietNky5y2yhtbUZ/MMGP1er
PDMyHz5MD9v78mstXXjx6WsBrOjc2a51WCufBV2drlPuwyTrstwJShbOhm1XfOld
1HSI9etIkwsVxBFgRfh6HUxbLGZWSHDZviY34KocxLNuuaRLqc5RHjTTZmzQwKl+
4EA4o6HJAYPsVz0Fu2/sJAy9cCclfCPy8Mcpstd1NIHOPhtNTpFiEQBl4zpXRqQs
nlmlNh1MhZepv84k+EvhRZCJlmfn0yLBLlNJfNh6jtdy6zv+lNf2oClrHDR9o5aL
nRCEJhRl/LX+BNf2mV5MULxAliXMME1qoYrm//BgsQk45t3XvVJ2fpPEVs8gcr49
gZ0yoJ20fcSW/UJCRMhoI9dtp1MYQmF3h3LHfNRROv+Ga9Uumw9OmQyFqFJwbF2o
doLOvDxtpEXWsiXcTUSqcl3CBGyT50AiAbRvv5yrQ5TU1Z8TplcUVUbJ/zifBoid
GexFUETsN3E1P0byvfbEiJJgaAYkWTaho2SwO+gRY/iieHlWyEbK6BwAiw7BsJOM
Tm5P6EjHyqk1kPiycoG9Zz70iiBhrFraMgzWwtrjNid2n8vvV6pdEpXpiaDzOAgS
SdVpHH2R8tu8GjnbONazSt6GpG7oEqnS5Nz2YZT6MMTcNuCvxY5bVoRfAl4DlDTM
X/98RH2w7xXM/fejAZN3
=Vjxf
-END PGP SIGNATURE-
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Remove gcc, gcc-c++ and make from minimal build root

[Petr Spacek]

On 15.1.2015 23:13, Nicolas Chauvet wrote:
 2015-01-15 20:18 GMT+01:00 Orion Poplawski or...@cora.nwra.com:
 
 On 01/15/2015 04:20 AM, Ralf Corsepius wrote:
 On 01/14/2015 03:10 AM, Orion Poplawski wrote:
 On 01/12/2015 06:08 AM, Vít Ondruch wrote:
 Dear Fedora developers,

 I'd like to collect some feedback about the $SUBJECT, i.e. making
 minimal build root really minimal, explicitly specifying build
 dependencies, etc.


 Would it be technically feasible to have a different buildroot for arch
 and noarch packages?
 What would this be useful for?

 The thought would be that (almost all) noarch packages don't need gcc, so
 the
 noarch buildroot could exclude gcc without impacting existing packages.
 
 
 I was going to say the same about noarch/arched packages and gcc
 assumption, also that I don't see any simplification in hardcoding the
 default compiler everywhere, specially as It probably depends on the build
 target . I remember other distros were using cross-compiler in buildroot
 and that was working pretty fine if the default compiler wasn't
 needlessly assumed.
 
 Another case about the default buildroot is compiler version, one could
 rely on a newer gcc (such as with a gcc5 package) and rebuild any packages
 with this new buildroot environment without tweaking any sources packages.

(I have no releng experience so I can be completely mistaken, please forgive
me :-))

Another advantage could be mass-rebuild simplification. Maybe we could save
machine and man-time by not rebuilding noarch packages because of gcc rebase
or something like that.

Maybe this can be somehow generalized: If we had a database with mapping:
SRPM - all packages in build root (implicit and explicit BuildRequires)
we could somehow limit mass rebuilds to packages affected by latest changes.

-- 
Petr^2 Spacek
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Marcin Juszkiewicz]

W dniu 16.01.2015 o 09:35, Petr Spacek pisze:
 Another advantage could be mass-rebuild simplification. Maybe we could save
 machine and man-time by not rebuilding noarch packages because of gcc rebase
 or something like that.

GCC change may affect binaries which will generate other output which
will change noarch packages.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Dridi Boukelmoune]

On Fri, Jan 16, 2015 at 10:08 AM, Marcin Juszkiewicz
mjuszkiew...@redhat.com wrote:
 GCC change may affect binaries which will generate other output which
 will change noarch packages.

It shouldn't change a program's behavior, unless the program itself is
relying on undefined behavior. Either way I would call that a bug.

The only output change that I could understand is something like a
`vim --version` (though it doesn't actually show gcc's version, but
close enough).
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Zbigniew Jędrzejewski-Szmek]

On Fri, Jan 16, 2015 at 11:19:13AM +0100, Dridi Boukelmoune wrote:
 On Fri, Jan 16, 2015 at 10:08 AM, Marcin Juszkiewicz
 mjuszkiew...@redhat.com wrote:
  GCC change may affect binaries which will generate other output which
  will change noarch packages.
 
 It shouldn't change a program's behavior, unless the program itself is
 relying on undefined behavior. Either way I would call that a bug.
Agreed. In principle, any package could affect the build of any other
package (e.g. bash version could realistically influence build results),
but we ignore this. As you say, something like this would happen only
if there was some (significant) bug. Those bugs are dealt with individually
when detected.

The same is true for the compiler version influencing the behaviour
of compiled programs. This might happen, but very rarely. Generally
updating the compiler should only result in changes in behaviour only
when the program was buggy to begin with and relied on undefined behaviour,
bad memory access, or similar.

On Fri, Jan 16, 2015 at 09:35:50AM +0100, Petr Spacek wrote:
 Another advantage could be mass-rebuild simplification. Maybe we could save
 machine and man-time by not rebuilding noarch packages because of gcc rebase
 or something like that.
+1

Only rebuilding packages based on explicit BR would be a nice optimization
for mass rebuilds.

 Maybe this can be somehow generalized: If we had a database with mapping:
 SRPM - all packages in build root (implicit and explicit BuildRequires)
 we could somehow limit mass rebuilds to packages affected by latest changes.

Zbyszek
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Miroslav Suchý]

On 01/14/2015 04:00 PM, Dennis Gilmore wrote:
 On 01/13/2015 06:01 PM, Dennis Gilmore wrote:
   that all being said. koji doesn't use any caching and will not use
   the lvm plugin. we make every buildroot from scratch using a fully
   clean environment to help with ensuring reproducability.
  
  You can cache and still preserve reproducability. What I'm planning
  for Copr is to do (every week/month) for chroot in fedora-20-x86_64
  fedora-21_86_64 ... ; do mock --init $chroot
done
  take snapshot of that. I plan to do that on VM level.
  And when new task come, I will just restore from that snapshot. And
  mock will start with already populated cache. So I will have better
  caching and yet reproducability.
 you really can't.  you would need to make a new cache any time one of
 the packages in the minimal buildroot changes.

That is why mock run 'yum update' on minimal buildroot before proceeding 
further.
So if one package in buildroot changes, you will just download and install just 
that one package and all other comes
from cache.

-- 
Miroslav Suchy, RHCE, RHCDS
Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Kevin Fenzi]

On Fri, 16 Jan 2015 14:53:36 +0100
Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl wrote:

 Agreed. In principle, any package could affect the build of any other
 package (e.g. bash version could realistically influence build
 results), but we ignore this. As you say, something like this would
 happen only if there was some (significant) bug. Those bugs are dealt
 with individually when detected.
 
 The same is true for the compiler version influencing the behaviour
 of compiled programs. This might happen, but very rarely. Generally
 updating the compiler should only result in changes in behaviour only
 when the program was buggy to begin with and relied on undefined
 behaviour, bad memory access, or similar.

Just to clarify here: mass rebuilds are done in a side tag. They
inherit the normal rawhide buildroot, and do not update it. So,
everything in the mass rebuild is built with (mostly) the same
buildroot. Things aren't added as they are rebuilt. Only at the end
when the side tag is merged back in is the rawhide buildroot updated
with all the new mass rebuilt builds. 

So, if packageA is mass rebuilt and that new build breaks building
packageB, we won't know of it until after the mass rebuild, both are
tagged in and the packageB maintainer tries to rebuild packageB for
some other reason. ;) 

I suppose someone with a lot of time on their hands and a good grasp of
graph theory could come up with a way to rebuild everything in an order
that would ensure that everything newly rebuilt was used to build the
next list. It would be much like bootstrapping a new arch. Also, I
suspect it would take a great deal more time than 40 hours. ;) 

kevin


pgp8rBnRNAzf2.pgp
Description: OpenPGP digital signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Dennis Gilmore]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Fri, 16 Jan 2015 17:18:17 +0100
Miroslav Suchý msu...@redhat.com wrote:

 On 01/14/2015 04:00 PM, Dennis Gilmore wrote:
  On 01/13/2015 06:01 PM, Dennis Gilmore wrote:
that all being said. koji doesn't use any caching and will
not use the lvm plugin. we make every buildroot from scratch
using a fully clean environment to help with ensuring
reproducability.
   
   You can cache and still preserve reproducability. What I'm
   planning for Copr is to do (every week/month) for chroot in
   fedora-20-x86_64 fedora-21_86_64 ... ; do mock --init $chroot
 done
   take snapshot of that. I plan to do that on VM level.
   And when new task come, I will just restore from that snapshot.
   And mock will start with already populated cache. So I will have
   better caching and yet reproducability.
  you really can't.  you would need to make a new cache any time one
  of the packages in the minimal buildroot changes.
 
 That is why mock run 'yum update' on minimal buildroot before
 proceeding further. So if one package in buildroot changes, you will
 just download and install just that one package and all other comes
 from cache.

That invalidates being able to reproduce the build root exactly. as you
would need to know and reproduce the package update. This is something
that has undergone a lot of thought and consideration and there is very
very good reasons things are the way they are. as i showed elsewhere in
this thread most repos are used only once or twice at most on any given
host.

Dennis
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBAgAGBQJUuceOAAoJEH7ltONmPFDRrq8QAIX07Q9MiY0RhXPveO+U48Kx
rPoU9N2Sbyje1pAphkDeTSlRqIJn/cFnHCorVPkbi2GEUbLZ/0bZSVixVr/7aIn0
BymonZJiuc5skQkA0WVOUZyK5O801dOrztzvMUEmTLGG6j5SSjFg3W4yO1Vnn1l3
P6vPnczjGDOn42mDjdKGUVnEifsryFlNIRkQQycaVCmGPh0vkJGX3cYzu6ALUS4F
ejSozB40LG6CMoLHDH4vi8MwLAz6/39siGxD/hkFOpziWSqAcvm6lSTlPREbSpTA
E7gY+FGaT+7jQDSpJViUcfrIAinFlIIab0VUkdI+zd0v7vK+NJBluwfGHBl1817m
IdA6Nz1jXG/5gQqTb8LdefCDuMyFpG1DWuiIF63vSmr83uMZOBSRWTtdS8jcRl39
qylx+CzKL4OAzBOlnSi437bitbyKOSLw2h0qDHiL29LvAsngKegLiXDiTvkGnkXX
zNWUTkaHg+BCg/qzl9Zz3dgEtXOfTHXHKWoRl8L2/i101W4ucgx5ZHQFMydo9jiN
e/e0E/FKqAAq0BbbK7+ltnKrpUeTxuQ74apjLbEAl6XMk9vYKORWxpUkHeGiasPM
EvhOPf5LEi+veCvRPuqzU0jVauJxJeaLY2w0vGkdqfoj1w6fqO6JR63BIv/tXC5A
MaD5XcAcF7a17uQDaQ1s
=4vDY
-END PGP SIGNATURE-
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Peter Robinson]

that all being said. koji doesn't use any caching and will
not use the lvm plugin. we make every buildroot from scratch
using a fully clean environment to help with ensuring
reproducability.
  
   You can cache and still preserve reproducability. What I'm
   planning for Copr is to do (every week/month) for chroot in
   fedora-20-x86_64 fedora-21_86_64 ... ; do mock --init $chroot
 done
   take snapshot of that. I plan to do that on VM level.
   And when new task come, I will just restore from that snapshot.
   And mock will start with already populated cache. So I will have
   better caching and yet reproducability.
  you really can't.  you would need to make a new cache any time one
  of the packages in the minimal buildroot changes.

 That is why mock run 'yum update' on minimal buildroot before
 proceeding further. So if one package in buildroot changes, you will
 just download and install just that one package and all other comes
 from cache.

 That invalidates being able to reproduce the build root exactly. as you
 would need to know and reproduce the package update. This is something
 that has undergone a lot of thought and consideration and there is very
 very good reasons things are the way they are. as i showed elsewhere in
 this thread most repos are used only once or twice at most on any given
 host.

The only way you could cache a base build root effectively is if you
track the packages and NVRs and when one of them bump you regenerate
the buildroot, likely pausing all builds until a new one is there.
there's likely not that much churn in the packages contained in the
root buildroot (you'd want to make sure you had everything for
building .src.rpms too) and in some cases you could likely get days on
the same image without having to rebuild it, other days you might have
to do it a few times a time.

Whether the effort to implement out weights the effort remains to be
seen. I think, given the low IOPs on the builds with an underlying COP
image it might well give a reasonable reduction in building especially
during mass rebuilds where the builders are very active and as it's in
a side tag the new build root packages don't end up in the build root
until tagged. Using fedmsg it's likely not hard either to monitor a
package subset for changes and regen the image.

Peter
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Kevin Kofler]

Vít Ondruch wrote:
 And somebody might be interested what is the trend:
 
 = F21
[snip]
 7320
[snip]
 = F20
[snip]
 6856
[snip]
 = F19
[snip]
 6609
[snip]
 = F18
[snip]
 6288

One can see that the number of compiled packages is actually INCREASING. 
It's just that the number of scripted packages is increasing even faster, 
which is probably because it is really easy to throw together a half-working 
quick hack in a scripting language and dumping it onto GitHub. (More and 
more upstreams these days don't even bother making releases!)

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Orion Poplawski]

On 01/15/2015 04:20 AM, Ralf Corsepius wrote:
 On 01/14/2015 03:10 AM, Orion Poplawski wrote:
 On 01/12/2015 06:08 AM, Vít Ondruch wrote:
 Dear Fedora developers,

 I'd like to collect some feedback about the $SUBJECT, i.e. making
 minimal build root really minimal, explicitly specifying build
 dependencies, etc.


 Would it be technically feasible to have a different buildroot for arch
 and noarch packages?
 What would this be useful for?

The thought would be that (almost all) noarch packages don't need gcc, so the
noarch buildroot could exclude gcc without impacting existing packages.


-- 
Orion Poplawski
Technical Manager 303-415-9701 x222
NWRA, Boulder/CoRA Office FAX: 303-415-9702
3380 Mitchell Lane   or...@nwra.com
Boulder, CO 80301   http://www.nwra.com
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Zbigniew Jędrzejewski-Szmek]

On Thu, Jan 15, 2015 at 11:13:17PM +0100, Nicolas Chauvet wrote:
 Another case about the default buildroot is compiler version, one could
 rely on a newer gcc (such as with a gcc5 package) and rebuild any packages
 with this new buildroot environment without tweaking any sources packages.
But you need to override packages which are explicitly BuildRequired
by source packages too. So a generic mechanism for overrides is needed
anyway, and can be used for the compiler too.

Zbyszek
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Nicolas Chauvet]

2015-01-15 20:18 GMT+01:00 Orion Poplawski or...@cora.nwra.com:

 On 01/15/2015 04:20 AM, Ralf Corsepius wrote:
  On 01/14/2015 03:10 AM, Orion Poplawski wrote:
  On 01/12/2015 06:08 AM, Vít Ondruch wrote:
  Dear Fedora developers,
 
  I'd like to collect some feedback about the $SUBJECT, i.e. making
  minimal build root really minimal, explicitly specifying build
  dependencies, etc.
 
 
  Would it be technically feasible to have a different buildroot for arch
  and noarch packages?
  What would this be useful for?

 The thought would be that (almost all) noarch packages don't need gcc, so
 the
 noarch buildroot could exclude gcc without impacting existing packages.


I was going to say the same about noarch/arched packages and gcc
assumption, also that I don't see any simplification in hardcoding the
default compiler everywhere, specially as It probably depends on the build
target . I remember other distros were using cross-compiler in buildroot
and that was working pretty fine if the default compiler wasn't
needlessly assumed.

Another case about the default buildroot is compiler version, one could
rely on a newer gcc (such as with a gcc5 package) and rebuild any packages
with this new buildroot environment without tweaking any sources packages.



-- 
-

Nicolas (kwizart)
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Igor Gnatenko]

On Mon, Jan 12, 2015 at 4:08 PM, Vít Ondruch vondr...@redhat.com wrote:
 Dear Fedora developers,
Hi,
 I'd like to collect some feedback about the $SUBJECT, i.e. making
 minimal build root really minimal, explicitly specifying build
 dependencies, etc.
I agree with removing this, but we need some time to investigate to
all packages.
 The topic originally spring out in this [1] discussion on packaging
 list. Interesting FPC discussion on this topic can be find in this
 ticket [2] and meeting minutes from last FPC meeting [3].

 Thanks


 Vít



 [1]
 https://lists.fedoraproject.org/pipermail/packaging/2014-December/010398.html
 [2] https://fedorahosted.org/fpc/ticket/490
 [3]
 https://lists.fedoraproject.org/pipermail/packaging/2015-January/010424.html


 --
 devel mailing list
 devel@lists.fedoraproject.org
 https://admin.fedoraproject.org/mailman/listinfo/devel
 Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct



-- 
-Igor Gnatenko
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Ralf Corsepius]

On 01/14/2015 03:10 AM, Orion Poplawski wrote:

On 01/12/2015 06:08 AM, Vít Ondruch wrote:

Dear Fedora developers,

I'd like to collect some feedback about the $SUBJECT, i.e. making
minimal build root really minimal, explicitly specifying build
dependencies, etc.



Would it be technically feasible to have a different buildroot for arch
and noarch packages?

What would this be useful for?

noarch-packages have dependencies on arch'ed packages (thus may trigger 
bugs inside of these packages), may generated arched files (because of 
packaging bugs) etc. pp.



 I.e. a buildsys-build-noarch group?
IMO, this is technically equivalent to building on one specific 
architecture, with the same pros and cons.


I.e. IMO all this would add is complexity.

Ralf

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Susi Lehtola]

On 01/14/2015 04:38 AM, Vít Ondruch wrote:

And it seems that this is the number of packages written in C++:

$ repoquery --disablerepo=* --enablerepo=rawhide --source --whatrequires
'libstdc++.so.6*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
2396

I'd like to point out at this place, that this would help also the 5006
packages written in C, since they don't need C++ to build. Only 14.8 %
of packages, which happens to be written in C++, would not benefit from
this change.


Instead of dropping make, gcc, and gcc-c++ from the minimal build root, 
would it make sense just to drop gcc-c++? That would only affect a small 
minority of packages, while eliminating some amount of additional 
packages in the buildroot.

--
Susi Lehtola
Fedora Project Contributor
jussileht...@fedoraproject.org
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Dennis Gilmore]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Wed, 14 Jan 2015 14:57:59 +0100
Miroslav Suchý msu...@redhat.com wrote:

 On 01/13/2015 06:01 PM, Dennis Gilmore wrote:
  that all being said. koji doesn't use any caching and will not use
  the lvm plugin. we make every buildroot from scratch using a fully
  clean environment to help with ensuring reproducability.
 
 You can cache and still preserve reproducability. What I'm planning
 for Copr is to do (every week/month) for chroot in fedora-20-x86_64
 fedora-21_86_64 ... ; do mock --init $chroot
   done
 take snapshot of that. I plan to do that on VM level.
 And when new task come, I will just restore from that snapshot. And
 mock will start with already populated cache. So I will have better
 caching and yet reproducability.

you really can't.  you would need to make a new cache any time one of
the packages in the minimal buildroot changes. while it may not be
intentional you can get differences in setup and behaviour due to
scriptlets doing different things on update and initial install. to get
reproducability you would need to start with the same packages set and
get the same updates. it really is not simple to do nor to track. which
is why we purposely disable all caching.

Dennis
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBAgAGBQJUtoSFAAoJEH7ltONmPFDR4ssP/RF2a1sQ1h+M7bU6A455311X
Q1JH7h6gcBLiYlcDfDURB2EoeQTqTkyqNgXRjNIZQc9B74/DprH2Tqipz/E3dvt2
i9oRL7yyO7uzaj+Epyg0Pa1ktCz4eEqvuTdUokgQtbwtK4nkgj7O5G9ui48vCbE2
bqsLzcavQqHb/UHT1yqBiEQaTptu07u8/KFRWwTI2HNAS8naQ+07/e26OkGT+NDZ
QT2nnVPPOAd21wps5Bi2r33CZzQHwQRnUd+XcXn0hARWy+9A06T4/4x0/N8C60Rf
NgDF0RyJouIilw4Y0YGijBPMY5aQ88YljVYm5UG8foGxi2romvW0DmpNumlB+1S+
gHS27G3EgY8zdlvZcIOlOR6TLRpfFBmGEMz9JrvNNoDaKEqBlBxt55AOBRw16oFQ
Oq9jdYMq1hlIjz58812yd/Qoc0RqreMU5UVX8NxLw0GLKUeDMTWyCU+jmqKWiWnC
99DiUOTjwncZL55wjXEgFAztL9XxcsJcFuzz3WJR84RI4OUNZbpyFlJ8u1/a5iaL
JYx1j6MrkVMBjtz3W8jOQ4218feB/kiGNbm37hH+GHTeuFPTIUu/dv/eE0LSMT0T
LFLSqe6toEZrlapIEmYEuHNpLnOPg1Y/0+mAEj6PEwVKWyXKDms53rICKq3y9dw2
2fhJJ4duyZKIYSfWA8KJ
=jxfx
-END PGP SIGNATURE-
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Gerd Hoffmann]

  Hi,

 I don't think the original question was all that great.  If we were to
 script the change to add the BRs, it would take time but it would
 mostly be a one-time cost.  Reducing buildroot time creation, network
 bandwidth, and storage savings is a reduction over each new buildroot
 instance creation over a long period of time.

IMO it isn't that clear there is a significant win due to the caching
mock is doing.  mock creates the minimal buildroot every once in a
while, then keeps reusing it for builds.  So, with a hot cache the build
goes like this:

  (1) unpack cached minimal buildroot
  (2) rpm install build deps
  (3) do the build

Taking gcc out of the minimal buildroot is an obvious win for packages
not using gcc.  But for packages using gcc it just moves gcc
installation from step (1) to step (2), and I expect this will slow down
the builds due to unpacking the cached buildroot being faster than doing
an rpm install.

So effectively we are going to trade faster non-gcc builds for slower
gcc builds.  Which still might be a net win at the end, but just from
looking at the proposal it isn't obvious whenever it actually is or not.

cheers,
  Gerd


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Miroslav Suchý]

On 01/13/2015 06:01 PM, Dennis Gilmore wrote:
 that all being said. koji doesn't use any caching and will not use the
 lvm plugin. we make every buildroot from scratch using a fully clean
 environment to help with ensuring reproducability.

You can cache and still preserve reproducability. What I'm planning for Copr is 
to do (every week/month)
  for chroot in fedora-20-x86_64 fedora-21_86_64 ... ; do
mock --init $chroot
  done
take snapshot of that. I plan to do that on VM level.
And when new task come, I will just restore from that snapshot. And mock will 
start with already populated cache. So I
will have better caching and yet reproducability.


-- 
Miroslav Suchy, RHCE, RHCDS
Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Vít Ondruch]

Dne 13.1.2015 v 18:20 Vít Ondruch napsal(a):
 Dne 13.1.2015 v 18:09 Vít Ondruch napsal(a):
 Dne 13.1.2015 v 17:54 Vít Ondruch napsal(a):
 So lets try to find what is written in C/C++ by some different way. Is
 that true, that every package in C/C++ compiled using gcc depends on
 glibc? Then we can use this query to get the number of packages:

 $ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)' | sort
 | uniq | wc -l
 2834
 The additional sed should remove the duplicated versions:

 $ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)' | sort
 -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
 2645

 And the wildcard in libc makes the difference :) So these seems to be
 numbers for Rawhide:

 $ repoquery --disablerepo=* --enablerepo=rawhide --source --whatrequires
 'libc.so.6*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
 7402

 $ dnf repoquery --disablerepo=* --enablerepo=rawhide-source --arch src
 '*' | wc -l
 16230

 I.e. 45.6 % of packages seems to be written in C/C++

And it seems that this is the number of packages written in C++:

$ repoquery --disablerepo=* --enablerepo=rawhide --source --whatrequires
'libstdc++.so.6*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
2396

I'd like to point out at this place, that this would help also the 5006
packages written in C, since they don't need C++ to build. Only 14.8 %
of packages, which happens to be written in C++, would not benefit from
this change.


Vít

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Richard W.M. Jones]

On Tue, Jan 13, 2015 at 04:26:00PM -0500, Miloslav Trmač wrote:
 (Incidentally, this would be much less of a problem if we had a way
 to add all the needed BuildRequires: automatically, and actually did
 that.  But the mere presence of these BuildRequires: in spec files
 would still be accidental complexity we are introducing into
 packaging—or alternatively, if we had a tool to detect this
 dependency then we wouldn’t need to explicitly list the dependency
 in the spec file.)

auto-buildrequires ...

BTW BR's are necessarily just needless complexity, since for
autoconf-using packages they control whether optional features of the
package are built.  Although arguably this is a problem in autoconf.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Richard W.M. Jones]

On Wed, Jan 14, 2015 at 12:45:39AM +, Richard W.M. Jones wrote:
 On Tue, Jan 13, 2015 at 04:26:00PM -0500, Miloslav Trmač wrote:
  (Incidentally, this would be much less of a problem if we had a way
  to add all the needed BuildRequires: automatically, and actually did
  that.  But the mere presence of these BuildRequires: in spec files
  would still be accidental complexity we are introducing into
  packaging—or alternatively, if we had a tool to detect this
  dependency then we wouldn’t need to explicitly list the dependency
  in the spec file.)
 
 auto-buildrequires ...
 
 BTW BR's are necessarily just needless complexity, since for
  ^not

 autoconf-using packages they control whether optional features of the
 package are built.  Although arguably this is a problem in autoconf.


-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Dennis Gilmore]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Tue, 13 Jan 2015 19:10:43 -0700
Orion Poplawski or...@cora.nwra.com wrote:

 On 01/12/2015 06:08 AM, Vít Ondruch wrote:
  Dear Fedora developers,
 
  I'd like to collect some feedback about the $SUBJECT, i.e. making
  minimal build root really minimal, explicitly specifying build
  dependencies, etc.
 
 
 Would it be technically feasible to have a different buildroot for
 arch and noarch packages?  I.e. a buildsys-build-noarch group?  Not
 sure how the koji/mock configs work here...

With changes to koji it is likely possible.

Dennis
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBAgAGBQJUteGRAAoJEH7ltONmPFDRaScP/jvgVl2m8nKnTbs91ZZ/Vn+U
SshuV6w7/TiP4fIKmLzmDDMD6frfH1FXcZ25Za7lW3UGzWfBx/4CgDZsVDprlNK0
AWNt4XcOYHQbgQ8aktJ9cIfNsRxWeSdWiNf+poc33s4q3cBXuRG7gQy+ph1ATboT
hFvpZpT+tbcPwl+1TI5I/0KTUkpjDMWNmaph8AVDE2FjayZpt+NlmdJ0XLLYt9E1
SZLIAteni+m5W2b2bqqplRQHTIWkQVZAmcFns1QX/7MpYHzttgllmq9vGsT2dAOR
eUtvh1W4YjMtRiNi0uOFazZoetVYKQdSTJc91hHw7X8sq6oy0PLmQo+egHZm4Cxd
48e55tWpUGi09ZrtlsXlpb+gnySkmL5fBw5zGSqrUlufwEEfqExJTu/Hzc26jXG7
E/MV3DA0t3eBP3N6n+TlHLbzabsAS41K42wPEsDX4szuw6DBlnPz5963s9TvV8Dl
AP4vHOTBpA1w0kXCQgvmv9HaOhIuRYcNJo4MCSF9lD3xYk1D36vcCVNHUV65g1G0
OwUkI1qFg+87GjJcjA8gwCWi7I3pkPPVT20R15MDiT/Mu4NjEq7VF6OEhjuucbfy
FxXnXaZkKR16lbqBESoc6HTCCY/EF6YqXVPkiuqRf4SxPwoqZdI1/fMcHGFWeic4
VboyWozD76wTu/Au+TrJ
=q5Ib
-END PGP SIGNATURE-
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Aleksandar Kurtakov]

- Original Message -
 From: Marian Csontos mcson...@redhat.com
 To: devel@lists.fedoraproject.org
 Sent: Tuesday, January 13, 2015 7:52:07 PM
 Subject: Re: Remove gcc, gcc-c++ and make from minimal build root
 
 On 01/13/2015 06:46 PM, Vít Ondruch wrote:
  Dne 13.1.2015 v 18:20 Vít Ondruch napsal(a):
  Dne 13.1.2015 v 18:09 Vít Ondruch napsal(a):
  Dne 13.1.2015 v 17:54 Vít Ondruch napsal(a):
  So lets try to find what is written in C/C++ by some different way. Is
  that true, that every package in C/C++ compiled using gcc depends on
  glibc? Then we can use this query to get the number of packages:
 
  $ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)' |
  sort
  | uniq | wc -l
  2834
  The additional sed should remove the duplicated versions:
 
  $ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)' | sort
  -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
  2645
 
  And the wildcard in libc makes the difference :) So these seems to be
  numbers for Rawhide:
 
  $ repoquery --disablerepo=* --enablerepo=rawhide --source --whatrequires
  'libc.so.6*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
  7402
 
  $ dnf repoquery --disablerepo=* --enablerepo=rawhide-source --arch src
  '*' | wc -l
  16230
 
  I.e. 45.6 % of packages seems to be written in C/C++
 
 You should try to turn that into number of spec files as it is not
 uncommon there are noarch packages built from same spec file as
 architecture dependent (like kernel-firmware.)

These are src.rpm (--arch src) so the mapping to spec files is 1:1.

Alex

 
 
 
  Vít
 
 
 
  And somebody might be interested what is the trend:
 
  = F21
 
  $ repoquery --disablerepo=* --enablerepo=fedora-source --arch src '*' |
  wc -l
  15842
 
  $ repoquery --disablerepo=* --enablerepo=fedora --source --whatrequires
  'libc.so.6*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
  7320
 
  46.2 %
 
  = F20
 
  $ repoquery --repoid=f
  --repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/20/Everything/source/SRPMS/'
  --arch=src '*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
  14364
 
  $ repoquery --repoid=f
  --repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/20/Everything/x86_64/os/'
  --source --whatrequires 'libc.so.6*' | sort -u | sed -r
  's/(.*)-.*-.*/\1/' | uniq | wc -l
  6856
 
  47.7 %
 
  = F19
 
  $ repoquery --repoid=f
  --repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/19/Everything/source/SRPMS/'
  --arch=src '*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
  13606
 
  $ repoquery --repoid=f
  --repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/19/Everything/x86_64/os/'
  --source --whatrequires 'libc.so.6*' | sort -u | sed -r
  's/(.*)-.*-.*/\1/' | uniq | wc -l
  6609
 
  48.6 %
 
  = F18
 
  $ repoquery --repoid=f
  --repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/18/Everything/source/SRPMS/'
  --arch=src '*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
  12614
 
  $ repoquery --repoid=f
  --repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/18/Everything/x86_64/os/'
  --source --whatrequires 'libc.so.6*' | sort -u | sed -r
  's/(.*)-.*-.*/\1/' | uniq | wc -l
  6288
 
  49.8%
 
 
 
  Vít
 
 
 
 --
 devel mailing list
 devel@lists.fedoraproject.org
 https://admin.fedoraproject.org/mailman/listinfo/devel
 Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Vít Ondruch]

Dne 13.1.2015 v 18:20 Vít Ondruch napsal(a):
 Dne 13.1.2015 v 18:09 Vít Ondruch napsal(a):
 Dne 13.1.2015 v 17:54 Vít Ondruch napsal(a):
 So lets try to find what is written in C/C++ by some different way. Is
 that true, that every package in C/C++ compiled using gcc depends on
 glibc? Then we can use this query to get the number of packages:

 $ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)' | sort
 | uniq | wc -l
 2834
 The additional sed should remove the duplicated versions:

 $ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)' | sort
 -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
 2645

 And the wildcard in libc makes the difference :) So these seems to be
 numbers for Rawhide:

 $ repoquery --disablerepo=* --enablerepo=rawhide --source --whatrequires
 'libc.so.6*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
 7402

 $ dnf repoquery --disablerepo=* --enablerepo=rawhide-source --arch src
 '*' | wc -l
 16230

 I.e. 45.6 % of packages seems to be written in C/C++


 Vít



And somebody might be interested what is the trend:

= F21

$ repoquery --disablerepo=* --enablerepo=fedora-source --arch src '*' |
wc -l
15842

$ repoquery --disablerepo=* --enablerepo=fedora --source --whatrequires
'libc.so.6*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
7320

46.2 %

= F20

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/20/Everything/source/SRPMS/'
--arch=src '*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
14364

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/20/Everything/x86_64/os/'
--source --whatrequires 'libc.so.6*' | sort -u | sed -r
's/(.*)-.*-.*/\1/' | uniq | wc -l
6856

47.7 %

= F19

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/19/Everything/source/SRPMS/'
--arch=src '*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
13606

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/19/Everything/x86_64/os/'
--source --whatrequires 'libc.so.6*' | sort -u | sed -r
's/(.*)-.*-.*/\1/' | uniq | wc -l
6609

48.6 %

= F18

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/18/Everything/source/SRPMS/'
--arch=src '*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
12614

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/18/Everything/x86_64/os/'
--source --whatrequires 'libc.so.6*' | sort -u | sed -r
's/(.*)-.*-.*/\1/' | uniq | wc -l
6288

49.8%



Vít


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Marian Csontos]

On 01/13/2015 06:53 PM, Aleksandar Kurtakov wrote:



- Original Message -

From: Marian Csontos mcson...@redhat.com
To: devel@lists.fedoraproject.org
Sent: Tuesday, January 13, 2015 7:52:07 PM
Subject: Re: Remove gcc, gcc-c++ and make from minimal build root

On 01/13/2015 06:46 PM, Vít Ondruch wrote:

Dne 13.1.2015 v 18:20 Vít Ondruch napsal(a):

Dne 13.1.2015 v 18:09 Vít Ondruch napsal(a):

Dne 13.1.2015 v 17:54 Vít Ondruch napsal(a):

So lets try to find what is written in C/C++ by some different way. Is
that true, that every package in C/C++ compiled using gcc depends on
glibc? Then we can use this query to get the number of packages:

$ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)' |
sort
| uniq | wc -l
2834

The additional sed should remove the duplicated versions:

$ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)' | sort
-u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
2645


And the wildcard in libc makes the difference :) So these seems to be
numbers for Rawhide:

$ repoquery --disablerepo=* --enablerepo=rawhide --source --whatrequires
'libc.so.6*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
7402

$ dnf repoquery --disablerepo=* --enablerepo=rawhide-source --arch src
'*' | wc -l
16230

I.e. 45.6 % of packages seems to be written in C/C++


You should try to turn that into number of spec files as it is not
uncommon there are noarch packages built from same spec file as
architecture dependent (like kernel-firmware.)


These are src.rpm (--arch src) so the mapping to spec files is 1:1.


Doh! You are of course right! Seems I have serious text understanding 
problems which reminds me it is past dinner time...


Sorry for noise.

-- Martian




Alex






Vít




And somebody might be interested what is the trend:

= F21

$ repoquery --disablerepo=* --enablerepo=fedora-source --arch src '*' |
wc -l
15842

$ repoquery --disablerepo=* --enablerepo=fedora --source --whatrequires
'libc.so.6*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
7320

46.2 %

= F20

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/20/Everything/source/SRPMS/'
--arch=src '*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
14364

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/20/Everything/x86_64/os/'
--source --whatrequires 'libc.so.6*' | sort -u | sed -r
's/(.*)-.*-.*/\1/' | uniq | wc -l
6856

47.7 %

= F19

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/19/Everything/source/SRPMS/'
--arch=src '*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
13606

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/19/Everything/x86_64/os/'
--source --whatrequires 'libc.so.6*' | sort -u | sed -r
's/(.*)-.*-.*/\1/' | uniq | wc -l
6609

48.6 %

= F18

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/18/Everything/source/SRPMS/'
--arch=src '*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
12614

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/18/Everything/x86_64/os/'
--source --whatrequires 'libc.so.6*' | sort -u | sed -r
's/(.*)-.*-.*/\1/' | uniq | wc -l
6288

49.8%



Vít




--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct


--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Vít Ondruch]

Dne 13.1.2015 v 17:54 Vít Ondruch napsal(a):
 So lets try to find what is written in C/C++ by some different way. Is
 that true, that every package in C/C++ compiled using gcc depends on
 glibc? Then we can use this query to get the number of packages:

 $ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)' | sort
 | uniq | wc -l
 2834

The additional sed should remove the duplicated versions:

$ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)' | sort
-u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
2645


Vít


 Well, ... any explanation why this figure differs significantly
 There are probably other reasons for the package to be arch dependent.
 First which comes to my mind is that somebody forgot to specify
 BuildArch: noarch.

 from this brute force estimate [1]:
 # rpm -q --qf %{SOURCERPM}\n \
  -p rawhide/i386/os/Packages/*/*.i686.rpm 2/dev/null | sort -u | wc -l
 7706
 Otherwise interesting number. I'd say this is top estimate of number of
 affected packages, since it probably suffers the same issue as mine
 query, i.e. there are the same source packages, with different versions,
 hence they are not filtered out as duplicates.

 Vít



 Ralf

 [1] Counting the unique src.rpms of all binary rpms in a local rawhide
 mirror.



-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Vít Ondruch]

Dne 13.1.2015 v 18:09 Vít Ondruch napsal(a):
 Dne 13.1.2015 v 17:54 Vít Ondruch napsal(a):
 So lets try to find what is written in C/C++ by some different way. Is
 that true, that every package in C/C++ compiled using gcc depends on
 glibc? Then we can use this query to get the number of packages:

 $ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)' | sort
 | uniq | wc -l
 2834
 The additional sed should remove the duplicated versions:

 $ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)' | sort
 -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
 2645


And the wildcard in libc makes the difference :) So these seems to be
numbers for Rawhide:

$ repoquery --disablerepo=* --enablerepo=rawhide --source --whatrequires
'libc.so.6*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
7402

$ dnf repoquery --disablerepo=* --enablerepo=rawhide-source --arch src
'*' | wc -l
16230

I.e. 45.6 % of packages seems to be written in C/C++


Vít


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Dennis Gilmore]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Tue, 13 Jan 2015 17:54:03 +0100
Vít Ondruch vondr...@redhat.com wrote:

 Dne 13.1.2015 v 17:42 Ralf Corsepius napsal(a):
  On 01/13/2015 12:06 PM, Vít Ondruch wrote:
  Dne 13.1.2015 v 08:12 Ralf Corsepius napsal(a):
 
  So we are even not able to tell how many packages are written in C
  or C++ in Fedora, since we do not have the BuildRequires speficied
  correctly.
  Correct.
 
  Because we have still the false feeling that everything is in
  C/C++, we regularly doing mass rebuilds and wasting resources (mass
  rebuild has also different advantages, but the rebuild due to
  rebased GCC is the major one).
  There are other reasons for mass-rebuilds. Most prominently: Changes
  inside of rpm and rpm-macros.
 
  So lets try to find what is written in C/C++ by some different
  way. Is that true, that every package in C/C++ compiled using gcc
  depends on glibc? Then we can use this query to get the number of
  packages:
 
  $ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)'
  | sort | uniq | wc -l
  2834
 
  Well, ... any explanation why this figure differs significantly
 
 There are probably other reasons for the package to be arch dependent.
 First which comes to my mind is that somebody forgot to specify
 BuildArch: noarch.
 
  from this brute force estimate [1]:
  # rpm -q --qf %{SOURCERPM}\n \
   -p rawhide/i386/os/Packages/*/*.i686.rpm 2/dev/null | sort -u |
  wc -l 7706
 
 Otherwise interesting number. I'd say this is top estimate of number
 of affected packages, since it probably suffers the same issue as mine
 query, i.e. there are the same source packages, with different
 versions, hence they are not filtered out as duplicates.

your assumption there is not possible. mash the tool that makes the
rawhide repo froma  koji tag only includes the latest build. so there
is only one srpm for each package and never multiples.  ie
foo-1.1-1.fc22 and foo-1.2-1.fc22 will never exist in the same repo.
there will never be multiples of the source rpms and never multiples of
the binaries. if a sub package gets dropped we do not keep the old
version it just goes away.

So I have some numbers

rpm -qp --qf %{arch}\n rawhide/source/SRPMS/*/*rpm|grep -v noarch |wc
- -l
7742
rpm -qp --qf %{arch}\n rawhide/source/SRPMS/*/*rpm|grep noarch |wc -l
8488

so right now we have 8488 packages in rawhide that are noarch only and
7742 that are arch specific so in theory just over 50% of builds would
get a slight build time speedup with removing gcc gcc-c++. I would like
to exclude make because I know a number of noarch only builds use make
and regular makefiles. that will take checking out the entire
sourcetree unpacking everything and doing analysis of the contents of
the prepped tree to determine.

Of the 7742 that are arch specific, we do not know how many need to
have gcc or gcc-c++ installed in the buildroot for them to compile.
though we could likely look at the BuildRequires and make an educated
guess.

Dennis
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBAgAGBQJUtVadAAoJEH7ltONmPFDR770P/i/7Yih5BSV7RHmkQ3wDRO0V
urSr01+Kxbs4HmgBgtstgbgHpHbU6H5zusNJkrcOZdyPeKMnn1bHigVkl0C8Ccq1
pb735ndpiSag64XbkXOCF7AgdKHJtBsU/GGhnsok7KgrqsZq+IhTUzYdKWAcOYdh
OiasG6Pg4TUCmz2KxyFCJBGnda67HTGYszkVUXptYpKtZefWIyUMr+Mgmr4hXuLV
l//GGAZTBy1E9S7L2WBcp8he/qWoNxhnxPonLYdS98Qa4i//fVjCP5kfEVs7jBNb
3CXl3q0al0K/sdxmxeLwZVicWDfer9gxBwGg1/FFJBrX6v8CTx3n/SEqexnyC4j6
N9u7Rz68mTxLIUhjd0p4gxA13PsJHv2jcPS8zVC32XElov9x+zbStrPv2oBAyi6C
y55ZW5ITggvXCm5EEC5EPeRHHKQtPi6AwTtsdmZRWL7dAGskRiJaei/bNktMD+BI
xe5JlVpxJcvX+y/rMyLsYr26CPLgmj1Rvv2e3wJjiPduaUUwGLfucqYyvuohYkEX
AmIQVmkO43mPwJbzYXrMxo2PVhygUT1iR110xZdRnFw0WtOHuh3JUNu6eEwtG4Hf
ZNHAOfMlit2BS/l8M+BMCW6L3ODLGO7nhnrmespliQnCyc2kaEQPweodsmJlnC2C
1Ci+Tgp/4juEojCmGP0P
=E+V1
-END PGP SIGNATURE-
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Marian Csontos]

On 01/13/2015 06:46 PM, Vít Ondruch wrote:

Dne 13.1.2015 v 18:20 Vít Ondruch napsal(a):

Dne 13.1.2015 v 18:09 Vít Ondruch napsal(a):

Dne 13.1.2015 v 17:54 Vít Ondruch napsal(a):

So lets try to find what is written in C/C++ by some different way. Is
that true, that every package in C/C++ compiled using gcc depends on
glibc? Then we can use this query to get the number of packages:

$ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)' | sort
| uniq | wc -l
2834

The additional sed should remove the duplicated versions:

$ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)' | sort
-u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
2645


And the wildcard in libc makes the difference :) So these seems to be
numbers for Rawhide:

$ repoquery --disablerepo=* --enablerepo=rawhide --source --whatrequires
'libc.so.6*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
7402

$ dnf repoquery --disablerepo=* --enablerepo=rawhide-source --arch src
'*' | wc -l
16230

I.e. 45.6 % of packages seems to be written in C/C++


You should try to turn that into number of spec files as it is not 
uncommon there are noarch packages built from same spec file as 
architecture dependent (like kernel-firmware.)





Vít




And somebody might be interested what is the trend:

= F21

$ repoquery --disablerepo=* --enablerepo=fedora-source --arch src '*' |
wc -l
15842

$ repoquery --disablerepo=* --enablerepo=fedora --source --whatrequires
'libc.so.6*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
7320

46.2 %

= F20

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/20/Everything/source/SRPMS/'
--arch=src '*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
14364

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/20/Everything/x86_64/os/'
--source --whatrequires 'libc.so.6*' | sort -u | sed -r
's/(.*)-.*-.*/\1/' | uniq | wc -l
6856

47.7 %

= F19

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/19/Everything/source/SRPMS/'
--arch=src '*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
13606

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/19/Everything/x86_64/os/'
--source --whatrequires 'libc.so.6*' | sort -u | sed -r
's/(.*)-.*-.*/\1/' | uniq | wc -l
6609

48.6 %

= F18

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/18/Everything/source/SRPMS/'
--arch=src '*' | sort -u | sed -r 's/(.*)-.*-.*/\1/' | uniq | wc -l
12614

$ repoquery --repoid=f
--repofrompath='f,http://download.fedoraproject.org/pub/fedora/linux/releases/18/Everything/x86_64/os/'
--source --whatrequires 'libc.so.6*' | sort -u | sed -r
's/(.*)-.*-.*/\1/' | uniq | wc -l
6288

49.8%



Vít




--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Vít Ondruch]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Dne 13.1.2015 v 18:01 Dennis Gilmore napsal(a):
 lets get some
 concrete data to look at.

So please, is there any chance to get similar data from Koji? You are
probably the best person who could provide something similar to my tests.

Thanks


Vít
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQIcBAEBAgAGBQJUtVSyAAoJEAzgnueZF7h8mfUQAJhtFcLtOOEMZ/ZDvFst5Iz1
8qAtOj0wmKzy2rnuE8wtRm/U+GvgVZbhF+083nla5dKKZY9dDyPyrIWuA7mbc7Sh
VeEF52eAKI2q5OwtYMjZNFKgSOVUqZ4h46PzTHoTpssg7Y13j4/QVnISKfGSix6K
9tK5KCRmEu9PgCmGmzaID7IX1GNPDy69A+VDG+cOd4my7fljJk8+mOCmKBPL1jWt
yJIPIB9xRH6fnadd41W5NxluV6xnUSI8PNjTv+Fc0z/IihRn3TXugRwRqIA99O/I
1W5aWj0+L+aRpPTmQPWm4T8oul13HQeFiWz6U1Zgw3Nh5OnYDhfa3oGVBBa00Xu+
0qg0U1OUysjRbClD9h9OLm2Km5mMfGZTCGbSoBZxYwWC851fp0oGlMQRfKDvCLTJ
0hcDx6KC1dTMcYGzlLGO4a+5hh8dJnTPAp6aLprZFnNYMEm+imPXeyvVjLmnGRxe
TSi/cGClqvaSF4qVnmG28WKFBdFvxiQhd7KGRCYlEtuIol4ANiY6Kz7kxgY1rWa0
lutdzjc93xqCyHXGVpyJSYzOXSj/SMcjc/Ku6aqE3oF6FjZ2hxAJGlAXQdRHy3ja
XvVq/BbIQy4pe4qtc07wF6ZfaG4YhtUzkynFhH/tYItldMeXWqPjJMEajWlwvOtL
eCBgOcnRlPRLHjTB60uz
=Y7Ei
-END PGP SIGNATURE-

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Christopher]

On Mon, Jan 12, 2015 at 7:35 PM, Kevin Kofler kevin.kof...@chello.at
wrote:

 Vít Ondruch wrote:
  I'd like to collect some feedback about the $SUBJECT, i.e. making
  minimal build root really minimal, explicitly specifying build
  dependencies, etc.

 -1, all the serious software requires gcc, gcc-c++ and make to build.


This is simply not true (depending on precisely what you mean by
serious). These days, many popular, important, and very relevant-to-users
packages are written in other languages than C/C++ (Java/Python, just to
name two). Please, let's not start disparaging packages written on other
languages as not serious software. I don't think that's going to be very
productive.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Christopher]

On Mon, Jan 12, 2015 at 8:37 AM, Mikolaj Izdebski mizde...@redhat.com
wrote:

 On 01/12/2015 02:08 PM, Vít Ondruch wrote:
  Dear Fedora developers,
 
  I'd like to collect some feedback about the $SUBJECT, i.e. making
  minimal build root really minimal, explicitly specifying build
  dependencies, etc.

 I strongly support this idea.

 In the past it probaby made sense to have GCC and friends in minimal
 buildroot, but nowdays we have packages written in dozen of different
 programming languages, which don't use GCC for compilation. It makes
 perfect sense to drop C-related build tools from Koji @bulid group.


I share this sentiment.

Additionally, I strongly prefer naming BuildRequires explicitly. It's
easier for contributors to understand what is going on, who can read a spec
file, but aren't familiar with the specific conventions of the distro. I'm
not a fan of magic dependencies.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Orion Poplawski]

On 01/12/2015 06:08 AM, Vít Ondruch wrote:

Dear Fedora developers,

I'd like to collect some feedback about the $SUBJECT, i.e. making
minimal build root really minimal, explicitly specifying build
dependencies, etc.



Would it be technically feasible to have a different buildroot for arch 
and noarch packages?  I.e. a buildsys-build-noarch group?  Not sure how 
the koji/mock configs work here...



--
Orion Poplawski
Technical Manager 303-415-9701 x222
NWRA/CoRA DivisionFAX: 303-415-9702
3380 Mitchell Lane  or...@cora.nwra.com
Boulder, CO 80301  http://www.cora.nwra.com
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Richard W.M. Jones]

On Tue, Jan 13, 2015 at 11:32:09AM -0600, Dennis Gilmore wrote:
 Of the 7742 that are arch specific, we do not know how many need to
 have gcc or gcc-c++ installed in the buildroot for them to compile.
 though we could likely look at the BuildRequires and make an educated
 guess.

auto-buildrequires can probably determine this.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Vít Ondruch]

Dne 13.1.2015 v 14:02 Richard W.M. Jones napsal(a):
 On Tue, Jan 13, 2015 at 01:31:06PM +0100, Sven Lankes wrote:
 On Tue, Jan 13, 2015 at 12:16:42PM +, Richard W.M. Jones wrote:

 That's simply an idiotic thing to say.  What is the progress involved
 in adding a new BR to thousands of packages?
 Am I wrong assuming that it would be rather easy to add the BR to all
 packages semi-automatically previous to a mass rebuild so that people
 who want to get rid of gcc for their packages can easily remove it?

 What am I missing?
 What is missing is there is no quantifcation yet of the benefit.
 How much faster will builds be?

Since you are asking I did 5 times:

$ mock -r minimal --scrub all  time mock -r minimal --init

Where the Total is DNF's output of package download speed and amount
of downloaded data obtained from mock output. Other DNF information are
shown on top of each section as well.


== with current minimal build root, i.e. install @buildsys-build

Install  164 Packages

Total download size: 119 M
Installed size: 445 M

1)
Total26 MB/s | 119 MB 00:04
real0m51.364s
user0m33.050s
sys0m6.454s

2)
Total13 MB/s | 119 MB 00:09
real0m51.479s
user0m33.024s
sys0m3.400s

3)
Total19 MB/s | 119 MB 00:06
real0m52.373s
user0m33.171s
sys0m7.121s

4)
Total21 MB/s | 119 MB 00:05
real0m54.213s
user0m32.999s
sys0m7.122s

5)
Total13 MB/s | 119 MB 00:09
real0m56.093s
user0m32.708s
sys0m5.745s


== without gcc, gcc-c++ and make, i.e. install rpm-build shadow-utils
util-linux which

Install  153 Packages

Total download size: 81 M
Installed size: 350 M

1)
Total14 MB/s |  81 MB 00:05
real0m49.789s
user0m28.926s
sys0m6.864s

2)
Total17 MB/s |  81 MB 00:04
real0m44.010s
user0m28.965s
sys0m3.434s

3)
Total18 MB/s |  81 MB 00:04
real0m45.860s
user0m29.032s
sys0m5.730s

4)
Total19 MB/s |  81 MB 00:04
real0m42.047s
user0m28.944s
sys0m3.099s

5)
Total14 MB/s |  81 MB 00:05
real0m47.079s
user0m29.839s
sys0m3.978s


== without gcc, gcc-c++, make and without perl (rhbz#1158860, note that
the rpm packages are local in this case), i.e. install rpm-build
shadow-utils util-linux which

Install  124 Packages

Total size: 69 M
Total download size: 68 M
Installed size: 310 M

1)
Total11 MB/s |  68 MB 00:06
real0m44.725s
user0m27.644s
sys0m6.126s

2)
Total12 MB/s |  68 MB 00:05
real0m46.253s
user0m27.624s
sys0m7.056s

3)
Total   6.5 MB/s |  68 MB 00:10
real0m50.296s
user0m27.567s
sys0m6.064s

4)
Total12 MB/s |  68 MB 00:05
real0m46.078s
user0m27.581s
sys0m7.698s

5)
Total16 MB/s |  68 MB 00:04
real0m44.307s
user0m27.830s
sys0m5.293s

== Only rpmbuild install install rpm-build shadow-utils

Install  111 Packages

Total size: 61 M
Total download size: 60 M
Installed size: 286 M

1)
Total20 MB/s |  60 MB 00:03
real0m37.848s
user0m26.158s
sys0m2.676s

2)
Total13 MB/s |  60 MB 00:04
real0m44.756s
user0m26.442s
sys0m5.185s

3)
Total16 MB/s |  60 MB 00:03
real0m42.703s
user0m26.118s
sys0m3.174s

4)
Total13 MB/s |  60 MB 00:04
real0m42.365s
user0m26.439s
sys0m4.193s

5)
Total   5.9 MB/s |  60 MB 00:10
real0m48.940s
user0m26.608s
sys0m4.576s

== Minimal root I can shell in, i.e. install bash shadow-utils

Install  48 Packages

Total download size: 31 M
Installed size: 176 M

1)
Total   8.3 MB/s |  31 MB 00:03
real0m36.745s
user0m22.019s
sys0m4.936s

2)
Total   9.5 MB/s |  31 MB 00:03
real0m32.608s
user0m21.805s
sys0m2.719s

3)
Total11 MB/s |  31 MB 00:02
real0m34.111s
user0m22.170s
sys0m3.651s

4)
Total12 MB/s |  31 MB 00:02
real0m41.141s
user0m22.595s
sys0m5.535s

5)
Total   8.4 MB/s |  31 MB 00:03

Re: Remove gcc, gcc-c++ and make from minimal build root

[Vít Ondruch]

Dne 13.1.2015 v 08:12 Ralf Corsepius napsal(a):
 On 01/13/2015 07:12 AM, Stanislav Ochotnicky wrote:
 On Tue 13 Jan 2015 01:35:26 AM CET Kevin Kofler wrote:
 Vít Ondruch wrote:
 I'd like to collect some feedback about the $SUBJECT, i.e. making
 minimal build root really minimal, explicitly specifying build
 dependencies, etc.

 -1, all the serious software requires gcc, gcc-c++ and make to build.

I am glad that you consider Ruby from ~66,6 % a serious software, since
it is written in C and uses make. Sadly, it is not written in mixture of
C and C++, otherwise it would reach 100 % seriousness achievement.

On the other hand, you probably don't consider any program/script
written in Ruby as serious, which decreases value/usefulness of Ruby to
zero. Hence your definition of serious software is doubtful.

Of course you can substitute Ruby for Python, Perl, Java 



 I actually think that cmake should be added to the minimal build root,
 instead of removing stuff. Almost all the packages I work on
 BuildRequire
 cmake (which also implies that they need make to build, and gcc-c++
 is the
 typical implementation language).

 Yes good idea. I worked on Java packages. Let's put Maven in minimal
 buildroot. I am sure everyone will enjoy it.

 Sorry for the sarcasm. Couldn't resist :-)

 Seriously though what's a problem with listing your package's real build
 requires?

 I basically see several issues:

 1. The sheer amount of packages being affect.

 Though I also think the ratio of scripted/compiled package has changed
 towards scripted languages, I do not think the number of compiled
 packages has decreased in absolute numbers and still is very high.

So we are even not able to tell how many packages are written in C or
C++ in Fedora, since we do not have the BuildRequires speficied
correctly. Because we have still the false feeling that everything is in
C/C++, we regularly doing mass rebuilds and wasting resources (mass
rebuild has also different advantages, but the rebuild due to rebased
GCC is the major one).

So lets try to find what is written in C/C++ by some different way. Is
that true, that every package in C/C++ compiled using gcc depends on
glibc? Then we can use this query to get the number of packages:

$ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)' | sort
| uniq | wc -l
2834

But there are different version of source packages, so the actual number
is lower.

I don't know how to determine the packages which are using make, unless
I want to checkout entire dist-git or analyze build logs.


 2. The semantics of such a change.

 To me, they are not clear, because having gcc or gcc-c++ in a
 build root covers multiple use-cases.
 For many packages, they just satisfy the need for an arbitrary
 c/c++ compiler, but actually do not mean a strict requirement to
 have gcc/g++. 

This is valid point. If we want to move to different compiler chain one
day, we would probably change the BR's. But:

1) This is not on the table any time soon.
2) It could be done in similar fashion to package renames, e.g. the new
compiler chain would declare the gcc provide.

 Similarly, they may exist packages which actually have dependencies on
 something which currently is being pulled in indirectly through gcc
 (say libgcc), while the package doesn't actually depend on a c-compiler.

That is fine with me. We will add BR the packages need. And you can
never prevent broken BR, since the packages you are building against are
evolving.



 3. There likely are many tiny problems under the hood (esp. in
 packages primarily written in scripted languages), such as (yet
 unknown and hidden) conditionally built
 features/sub-components/sub-packages and conditional deps etc.

Definitely, something will bite us. Nobody claims the opposite. Nobody
says its piece of cake, lets do it for F22.



 I.e. though I am in favor of minimising the buildroots, I don't expect
 removing gcc/g++ will become an easy effort nor should it be taken
 light-heartedly.

I proposed several steps:

1) Update packaging guidelines, get rid of the exceptions (at least the
gcc, gcc-c++ and make), let developers specify these dependencies
explicitly.
2) Update rpmlint to fire warning when suitable
3) Drop the gcc, gcc-c++ and make from minimal build root in F24 timeframe

Of course we can do 1 and 2 and reevaluate 3 anytime in the future.


Vít

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Helio Chissini de Castro]

The arguments on change the compiler not near enough is damn failed as well.
LLVM and now clang are growing towards a been a standard soon for many
applications.

Helio

On Tue, Jan 13, 2015 at 10:16 AM, Richard W.M. Jones rjo...@redhat.com
wrote:

 On Mon, Jan 12, 2015 at 02:38:44PM +0100, Mikolaj Izdebski wrote:
  On 01/12/2015 02:30 PM, Marcin Juszkiewicz wrote:
   W dniu 12.01.2015 o 14:08, Vít Ondruch pisze:
  
   I'd like to collect some feedback about the $SUBJECT, i.e. making
   minimal build root really minimal, explicitly specifying build
   dependencies, etc.
  
   Are you aware that this means changing build dependencies of most of
   packages? Altering them just to add gcc, make etc.
 
  There is no progress without change.

 That's simply an idiotic thing to say.  What is the progress involved
 in adding a new BR to thousands of packages?

 Rich.

 --
 Richard Jones, Virtualization Group, Red Hat
 http://people.redhat.com/~rjones
 Read my programming and virtualization blog: http://rwmj.wordpress.com
 virt-builder quickly builds VMs from scratch
 http://libguestfs.org/virt-builder.1.html
 --
 devel mailing list
 devel@lists.fedoraproject.org
 https://admin.fedoraproject.org/mailman/listinfo/devel
 Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Richard W.M. Jones]

On Mon, Jan 12, 2015 at 02:38:44PM +0100, Mikolaj Izdebski wrote:
 On 01/12/2015 02:30 PM, Marcin Juszkiewicz wrote:
  W dniu 12.01.2015 o 14:08, Vít Ondruch pisze:
  
  I'd like to collect some feedback about the $SUBJECT, i.e. making 
  minimal build root really minimal, explicitly specifying build 
  dependencies, etc.
  
  Are you aware that this means changing build dependencies of most of
  packages? Altering them just to add gcc, make etc.
 
 There is no progress without change.

That's simply an idiotic thing to say.  What is the progress involved
in adding a new BR to thousands of packages?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Sven Lankes]

On Tue, Jan 13, 2015 at 12:16:42PM +, Richard W.M. Jones wrote:

 That's simply an idiotic thing to say.  What is the progress involved
 in adding a new BR to thousands of packages?

Am I wrong assuming that it would be rather easy to add the BR to all
packages semi-automatically previous to a mass rebuild so that people
who want to get rid of gcc for their packages can easily remove it?

What am I missing?

-- 
sven === jabber/xmpp: s...@lankes.net
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Richard W.M. Jones]

On Tue, Jan 13, 2015 at 01:31:06PM +0100, Sven Lankes wrote:
 On Tue, Jan 13, 2015 at 12:16:42PM +, Richard W.M. Jones wrote:
 
  That's simply an idiotic thing to say.  What is the progress involved
  in adding a new BR to thousands of packages?
 
 Am I wrong assuming that it would be rather easy to add the BR to all
 packages semi-automatically previous to a mass rebuild so that people
 who want to get rid of gcc for their packages can easily remove it?
 
 What am I missing?

What is missing is there is no quantifcation yet of the benefit.
How much faster will builds be?  Versus how long will it take to
either modify all these packages / write and run the script to do
that?  Is A  B?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Vít Ondruch]

Forgot to add this was measured on my Lenovo T440s, with SSD and 12 GB
RAM, running F21 with mock 1.2.3, configured to use LVM plugin and DNF
(see attached config files). And network? Who knows, probably fast.
Should be 1Gb/s if I am not mistaken.

Not sure how similar/different the tests could look on Fedoras Koji
builders.


Vít



Dne 13.1.2015 v 15:56 Vít Ondruch napsal(a):
 Dne 13.1.2015 v 14:02 Richard W.M. Jones napsal(a):
 On Tue, Jan 13, 2015 at 01:31:06PM +0100, Sven Lankes wrote:
 On Tue, Jan 13, 2015 at 12:16:42PM +, Richard W.M. Jones wrote:

 That's simply an idiotic thing to say.  What is the progress involved
 in adding a new BR to thousands of packages?
 Am I wrong assuming that it would be rather easy to add the BR to all
 packages semi-automatically previous to a mass rebuild so that people
 who want to get rid of gcc for their packages can easily remove it?

 What am I missing?
 What is missing is there is no quantifcation yet of the benefit.
 How much faster will builds be?

 Since you are asking I did 5 times:

 $ mock -r minimal --scrub all  time mock -r minimal --init

 Where the Total is DNF's output of package download speed and amount
 of downloaded data obtained from mock output. Other DNF information
 are shown on top of each section as well.


 == with current minimal build root, i.e. install @buildsys-build

 Install  164 Packages

 Total download size: 119 M
 Installed size: 445 M

 1)
 Total26 MB/s | 119 MB
 00:04
 real0m51.364s
 user0m33.050s
 sys0m6.454s

 2)
 Total13 MB/s | 119 MB
 00:09
 real0m51.479s
 user0m33.024s
 sys0m3.400s

 3)
 Total19 MB/s | 119 MB
 00:06
 real0m52.373s
 user0m33.171s
 sys0m7.121s

 4)
 Total21 MB/s | 119 MB
 00:05
 real0m54.213s
 user0m32.999s
 sys0m7.122s

 5)
 Total13 MB/s | 119 MB
 00:09
 real0m56.093s
 user0m32.708s
 sys0m5.745s


 == without gcc, gcc-c++ and make, i.e. install rpm-build shadow-utils
 util-linux which

 Install  153 Packages

 Total download size: 81 M
 Installed size: 350 M

 1)
 Total14 MB/s |  81 MB
 00:05
 real0m49.789s
 user0m28.926s
 sys0m6.864s

 2)
 Total17 MB/s |  81 MB
 00:04
 real0m44.010s
 user0m28.965s
 sys0m3.434s

 3)
 Total18 MB/s |  81 MB
 00:04
 real0m45.860s
 user0m29.032s
 sys0m5.730s

 4)
 Total19 MB/s |  81 MB
 00:04
 real0m42.047s
 user0m28.944s
 sys0m3.099s

 5)
 Total14 MB/s |  81 MB
 00:05
 real0m47.079s
 user0m29.839s
 sys0m3.978s


 == without gcc, gcc-c++, make and without perl (rhbz#1158860, note
 that the rpm packages are local in this case), i.e. install rpm-build
 shadow-utils util-linux which

 Install  124 Packages

 Total size: 69 M
 Total download size: 68 M
 Installed size: 310 M

 1)
 Total11 MB/s |  68 MB
 00:06
 real0m44.725s
 user0m27.644s
 sys0m6.126s

 2)
 Total12 MB/s |  68 MB
 00:05
 real0m46.253s
 user0m27.624s
 sys0m7.056s

 3)
 Total   6.5 MB/s |  68 MB
 00:10
 real0m50.296s
 user0m27.567s
 sys0m6.064s

 4)
 Total12 MB/s |  68 MB
 00:05
 real0m46.078s
 user0m27.581s
 sys0m7.698s

 5)
 Total16 MB/s |  68 MB
 00:04
 real0m44.307s
 user0m27.830s
 sys0m5.293s

 == Only rpmbuild install install rpm-build shadow-utils

 Install  111 Packages

 Total size: 61 M
 Total download size: 60 M
 Installed size: 286 M

 1)
 Total20 MB/s |  60 MB
 00:03
 real0m37.848s
 user0m26.158s
 sys0m2.676s

 2)
 Total13 MB/s |  60 MB
 00:04
 real0m44.756s
 user0m26.442s
 sys0m5.185s

 3)
 Total16 MB/s |  60 MB
 00:03
 real0m42.703s
 user0m26.118s
 sys0m3.174s

 4)
 Total13 MB/s |  60 MB
 00:04
 real0m42.365s
 user0m26.439s
 sys0m4.193s

 5)
 Total   5.9 MB/s |  60 MB
 00:10
 real0m48.940s
 user0m26.608s
 sys0m4.576s

 == Minimal root I can shell in, i.e. install bash shadow-utils

 Install  48 Packages

 Total download size: 31 M
 Installed size: 176 M

 1)
 Total   8.3 MB/s |  31 MB
 

Re: Remove gcc, gcc-c++ and make from minimal build root

[Josh Boyer]

On Tue, Jan 13, 2015 at 9:56 AM, Vít Ondruch vondr...@redhat.com wrote:
 Dne 13.1.2015 v 14:02 Richard W.M. Jones napsal(a):

 On Tue, Jan 13, 2015 at 01:31:06PM +0100, Sven Lankes wrote:

 On Tue, Jan 13, 2015 at 12:16:42PM +, Richard W.M. Jones wrote:

 That's simply an idiotic thing to say.  What is the progress involved
 in adding a new BR to thousands of packages?

 Am I wrong assuming that it would be rather easy to add the BR to all
 packages semi-automatically previous to a mass rebuild so that people
 who want to get rid of gcc for their packages can easily remove it?

 What am I missing?

 What is missing is there is no quantifcation yet of the benefit.
 How much faster will builds be?


 Since you are asking I did 5 times:

 $ mock -r minimal --scrub all  time mock -r minimal --init

 Where the Total is DNF's output of package download speed and amount of
 downloaded data obtained from mock output. Other DNF information are shown
 on top of each section as well.


 == with current minimal build root, i.e. install @buildsys-build

 Install  164 Packages

 Total download size: 119 M
 Installed size: 445 M

 1)
 Total26 MB/s | 119 MB 00:04
 real0m51.364s
 user0m33.050s
 sys0m6.454s

 2)
 Total13 MB/s | 119 MB 00:09
 real0m51.479s
 user0m33.024s
 sys0m3.400s

 3)
 Total19 MB/s | 119 MB 00:06
 real0m52.373s
 user0m33.171s
 sys0m7.121s

 4)
 Total21 MB/s | 119 MB 00:05
 real0m54.213s
 user0m32.999s
 sys0m7.122s

 5)
 Total13 MB/s | 119 MB 00:09
 real0m56.093s
 user0m32.708s
 sys0m5.745s


 == without gcc, gcc-c++ and make, i.e. install rpm-build shadow-utils
 util-linux which

 Install  153 Packages

 Total download size: 81 M
 Installed size: 350 M

 1)
 Total14 MB/s |  81 MB 00:05
 real0m49.789s
 user0m28.926s
 sys0m6.864s

 2)
 Total17 MB/s |  81 MB 00:04
 real0m44.010s
 user0m28.965s
 sys0m3.434s

 3)
 Total18 MB/s |  81 MB 00:04
 real0m45.860s
 user0m29.032s
 sys0m5.730s

 4)
 Total19 MB/s |  81 MB 00:04
 real0m42.047s
 user0m28.944s
 sys0m3.099s

 5)
 Total14 MB/s |  81 MB 00:05
 real0m47.079s
 user0m29.839s
 sys0m3.978s


 == without gcc, gcc-c++, make and without perl (rhbz#1158860, note that the
 rpm packages are local in this case), i.e. install rpm-build shadow-utils
 util-linux which

 Install  124 Packages

 Total size: 69 M
 Total download size: 68 M
 Installed size: 310 M

 1)
 Total11 MB/s |  68 MB 00:06
 real0m44.725s
 user0m27.644s
 sys0m6.126s

 2)
 Total12 MB/s |  68 MB 00:05
 real0m46.253s
 user0m27.624s
 sys0m7.056s

 3)
 Total   6.5 MB/s |  68 MB 00:10
 real0m50.296s
 user0m27.567s
 sys0m6.064s

 4)
 Total12 MB/s |  68 MB 00:05
 real0m46.078s
 user0m27.581s
 sys0m7.698s

 5)
 Total16 MB/s |  68 MB 00:04
 real0m44.307s
 user0m27.830s
 sys0m5.293s

 == Only rpmbuild install install rpm-build shadow-utils

 Install  111 Packages

 Total size: 61 M
 Total download size: 60 M
 Installed size: 286 M

 1)
 Total20 MB/s |  60 MB 00:03
 real0m37.848s
 user0m26.158s
 sys0m2.676s

 2)
 Total13 MB/s |  60 MB 00:04
 real0m44.756s
 user0m26.442s
 sys0m5.185s

 3)
 Total16 MB/s |  60 MB 00:03
 real0m42.703s
 user0m26.118s
 sys0m3.174s

 4)
 Total13 MB/s |  60 MB 00:04
 real0m42.365s
 user0m26.439s
 sys0m4.193s

 5)
 Total   5.9 MB/s |  60 MB 00:10
 real0m48.940s
 user0m26.608s
 sys0m4.576s

 == Minimal root I can shell in, i.e. install bash shadow-utils

 Install  48 Packages

 Total download size: 31 M
 Installed size: 176 M

 1)
 Total   8.3 MB/s |  31 MB 00:03
 real0m36.745s
 user0m22.019s
 sys0m4.936s

 2)
 Total   9.5 MB/s |  31 MB 00:03
 real0m32.608s
 user0m21.805s
 sys0m2.719s

 3)
 Total11 MB/s |  31 MB 00:02
 real0m34.111s
 user0m22.170s
 sys

Re: Remove gcc, gcc-c++ and make from minimal build root

[Peter Robinson]

 You can interpret these data yourself, but with less packages in build root,
 I can see:

 1) Saved build time
 2) Saved network bandwidth
 3) Saved storage
 4) Less things to break

 Just to make sure I'm reading the data correctly, your tests show a
 maximum approximate savings of 20 seconds going from the current
 buildroot to the minimal shell root?

That's on a local laptop with a SSD so hardly indicative of the
standard koji build env.

   Versus how long will it take to
 either modify all these packages / write and run the script to do
 that?  Is A  B?


 There are things which are hard to quantify. If you want to prohibit any
 change, this is probably right question to ask. Of course answering such
 questions also takes some time, I hope you will count it into the final
 equation at the end, although hard to say on which side it should belong.

 I don't think the original question was all that great.  If we were to
 script the change to add the BRs, it would take time but it would
 mostly be a one-time cost.  Reducing buildroot time creation, network
 bandwidth, and storage savings is a reduction over each new buildroot
 instance creation over a long period of time.  And as you said, you
 could also wind up saving time for issues where the buildroot doesn't
 actually install because of broken packages (e.g. a broken perl dep or
 something).  The question, as posed, wasn't really taking everything
 into account.

 That being said, 20 seconds isn't saving very much.  The network
 bandwidth savings exists, but given that mock caches packages it isn't
 a savings for every build.  The storage savings I'm not sure I really
 agree with because mock roots are transient things and you can delete
 them if you need to.  You only wind up saving a few megabytes.  I'm
 not sure this is compelling enough data at the moment.

And the new mock does have the ability to use a pre created COW build
root off local disk (I've not looked closely at exact implementation
deteails) which should actually remove the vast majority of all of the
above savings and make most of this a mute point. I'm not sure what
extra work needs to be done to make that standard in koji though.

Peter
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Dridi Boukelmoune]

On Mon, Jan 12, 2015 at 2:08 PM, Vít Ondruch vondr...@redhat.com wrote:
 Dear Fedora developers,

 I'd like to collect some feedback about the $SUBJECT, i.e. making
 minimal build root really minimal, explicitly specifying build
 dependencies, etc.

Hi,

I think this is a good idea, because overall I tend to avoid implicit
or invisible rules. And the man says mock - build SRPMs in a
chroot so having a minimal tree in which you can chroot and find
tools to build SRPMs sounds like a neat idea :)

 The topic originally spring out in this [1] discussion on packaging
 list. Interesting FPC discussion on this topic can be find in this
 ticket [2] and meeting minutes from last FPC meeting [3].

As a mock user on RHEL and CentOS systems, I haven't seen any mention
of EPEL in the FPC ticket and minutes. I may be wrong, but I'm pretty
sure I got mock and the @buildsys-build group from the EPEL repos. You
might want to enable such a change for the next EPEL too, or at least
take it into account.

I don't have access to those machines right now.

Dridi

 Thanks


 Vít



 [1]
 https://lists.fedoraproject.org/pipermail/packaging/2014-December/010398.html
 [2] https://fedorahosted.org/fpc/ticket/490
 [3]
 https://lists.fedoraproject.org/pipermail/packaging/2015-January/010424.html


 --
 devel mailing list
 devel@lists.fedoraproject.org
 https://admin.fedoraproject.org/mailman/listinfo/devel
 Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Vít Ondruch]

Dne 13.1.2015 v 16:35 Peter Robinson napsal(a):
 You can interpret these data yourself, but with less packages in build root,
 I can see:

 1) Saved build time
 2) Saved network bandwidth
 3) Saved storage
 4) Less things to break
 Just to make sure I'm reading the data correctly, your tests show a
 maximum approximate savings of 20 seconds going from the current
 buildroot to the minimal shell root?
 That's on a local laptop with a SSD so hardly indicative of the
 standard koji build env.

Right, relengs/infrastructure guys are the only one who could give us
numbers for Koji. But we should not focus just on Koji, packagers are
very important asset as well. They provide their own resources to Fedora.


   Versus how long will it take to
 either modify all these packages / write and run the script to do
 that?  Is A  B?


 There are things which are hard to quantify. If you want to prohibit any
 change, this is probably right question to ask. Of course answering such
 questions also takes some time, I hope you will count it into the final
 equation at the end, although hard to say on which side it should belong.
 I don't think the original question was all that great.  If we were to
 script the change to add the BRs, it would take time but it would
 mostly be a one-time cost.  Reducing buildroot time creation, network
 bandwidth, and storage savings is a reduction over each new buildroot
 instance creation over a long period of time.  And as you said, you
 could also wind up saving time for issues where the buildroot doesn't
 actually install because of broken packages (e.g. a broken perl dep or
 something).  The question, as posed, wasn't really taking everything
 into account.

 That being said, 20 seconds isn't saving very much.  The network
 bandwidth savings exists, but given that mock caches packages it isn't
 a savings for every build.  The storage savings I'm not sure I really
 agree with because mock roots are transient things and you can delete
 them if you need to.  You only wind up saving a few megabytes.  I'm
 not sure this is compelling enough data at the moment.
 And the new mock does have the ability to use a pre created COW build
 root off local disk (I've not looked closely at exact implementation
 deteails) which should actually remove the vast majority of all of the
 above savings and make most of this a mute point. I'm not sure what
 extra work needs to be done to make that standard in koji though.

Well, I used to be more optimistic about LVM snapshots prior I tried it
myself :/ But it should be definitely more useful for Koji then for my
local builds. I highly suggest everybody to at least try the LVM plugin.

Vít

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Ralf Corsepius]

On 01/13/2015 12:06 PM, Vít Ondruch wrote:

Dne 13.1.2015 v 08:12 Ralf Corsepius napsal(a):

So we are even not able to tell how many packages are written in C or
C++ in Fedora, since we do not have the BuildRequires speficied
correctly.

Correct.


Because we have still the false feeling that everything is in
C/C++, we regularly doing mass rebuilds and wasting resources (mass
rebuild has also different advantages, but the rebuild due to rebased
GCC is the major one).
There are other reasons for mass-rebuilds. Most prominently: Changes 
inside of rpm and rpm-macros.



So lets try to find what is written in C/C++ by some different way. Is
that true, that every package in C/C++ compiled using gcc depends on
glibc? Then we can use this query to get the number of packages:

$ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)' | sort
| uniq | wc -l
2834


Well, ... any explanation why this figure differs significantly from 
this brute force estimate [1]:

# rpm -q --qf %{SOURCERPM}\n \
 -p rawhide/i386/os/Packages/*/*.i686.rpm 2/dev/null | sort -u | wc -l
7706

Ralf

[1] Counting the unique src.rpms of all binary rpms in a local rawhide 
mirror.



--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Dennis Gilmore]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Mon, 12 Jan 2015 14:08:24 +0100
Vít Ondruch vondr...@redhat.com wrote:

 Dear Fedora developers,
 
 I'd like to collect some feedback about the $SUBJECT, i.e. making
 minimal build root really minimal, explicitly specifying build
 dependencies, etc.
 
 The topic originally spring out in this [1] discussion on packaging
 list. Interesting FPC discussion on this topic can be find in this
 ticket [2] and meeting minutes from last FPC meeting [3].

As I have stated in a few forums a much bigger win is switching fedpkg
with fedpkg minimal in the srpm-build group in koji that will result in
a much much smaller buildroot to make the srpm and work is underway to
make that happen. at least in terms of generation times of the
buildroots during the koji tasks.

I also do not think make should be removed. I could see a case for gcc
gcc-c++  but it needs further analysis of the 15k odd packages in
fedora to see how many would need to add gcc or gcc-c++ to the
BuildRequires and how many packages would get a slightly faster
buildroot due to not needing gcc or gcc-g++ and having a few less
packages installed.

Dennis
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBAgAGBQJUtUq3AAoJEH7ltONmPFDRr+UP/0hvkuAJKah3ANk30T9r35ru
XbF0izIvdH2Mm63uvJ8NdFfuUoQS+T9HgR09i8ENfzXThUcXVqn3jgNDHBjT6F+H
3i4v2U8u9vdkGGQ1zcZPwiPpXM/EqXs7RyaClsE5WWGNfPVWzzcxwFEtxbp3Bu8e
avGEAL+84x8T10lEQZi1AegkQqjsV5lvoyzjyaRF3Dn+Jt0pLsNFclCQp+SLNSNd
M8LygRF+xTm8ZdHzd8z99C71f3jOgLA2Rx6+2YaBM2pRqDANolL1XDY9ebDxOZVP
8v4KLqHNXr/NaR0ZC2i/Tso3H4+XnJPzGBceziHnbct0u66sH4FENo4g/RIsYCMh
S/GDzUSO+w/At9XRhqWXdOGmz0Q/aMW2H4AhafoF6sYONc+L70VqUTVQGfHcoJtT
/to2Q+8G7J2B/34rt6r6sDBKhmV7jsOoKkoYFXBN58PO3TM8jFOvW8wKIXjfMhm5
6w8B8JsT5YcUkUBljXvOpmsJnZo9OdLiXtfaOaeJVqHcCQCODCwc/zvKnR/sNpEr
+HdHx/DRdQjH2VBdlOYHmzM+xjtjub1SEbyUlQWaFz+/6Ls2iIHclMrY1zrH+899
KCl1gjaw62iulfFAGck04ul925pfPDH9sLec/1LSoM3sBpYE0Dk6QlL4ZnZHqMnz
xQfRXfbSm663Y3hehK7/
=07I8
-END PGP SIGNATURE-
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Vít Ondruch]

Dne 13.1.2015 v 17:42 Ralf Corsepius napsal(a):
 On 01/13/2015 12:06 PM, Vít Ondruch wrote:
 Dne 13.1.2015 v 08:12 Ralf Corsepius napsal(a):

 So we are even not able to tell how many packages are written in C or
 C++ in Fedora, since we do not have the BuildRequires speficied
 correctly.
 Correct.

 Because we have still the false feeling that everything is in
 C/C++, we regularly doing mass rebuilds and wasting resources (mass
 rebuild has also different advantages, but the rebuild due to rebased
 GCC is the major one).
 There are other reasons for mass-rebuilds. Most prominently: Changes
 inside of rpm and rpm-macros.

 So lets try to find what is written in C/C++ by some different way. Is
 that true, that every package in C/C++ compiled using gcc depends on
 glibc? Then we can use this query to get the number of packages:

 $ repoquery --source --whatrequires 'libc.so.6(GLIBC_2.4)(64bit)' | sort
 | uniq | wc -l
 2834

 Well, ... any explanation why this figure differs significantly

There are probably other reasons for the package to be arch dependent.
First which comes to my mind is that somebody forgot to specify
BuildArch: noarch.

 from this brute force estimate [1]:
 # rpm -q --qf %{SOURCERPM}\n \
  -p rawhide/i386/os/Packages/*/*.i686.rpm 2/dev/null | sort -u | wc -l
 7706

Otherwise interesting number. I'd say this is top estimate of number of
affected packages, since it probably suffers the same issue as mine
query, i.e. there are the same source packages, with different versions,
hence they are not filtered out as duplicates.

Vít




 Ralf

 [1] Counting the unique src.rpms of all binary rpms in a local rawhide
 mirror.



-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Dennis Gilmore]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Tue, 13 Jan 2015 10:28:36 -0500
Josh Boyer jwbo...@fedoraproject.org wrote:

 On Tue, Jan 13, 2015 at 9:56 AM, Vít Ondruch vondr...@redhat.com
 wrote:
  Dne 13.1.2015 v 14:02 Richard W.M. Jones napsal(a):
 
  On Tue, Jan 13, 2015 at 01:31:06PM +0100, Sven Lankes wrote:
 
  On Tue, Jan 13, 2015 at 12:16:42PM +, Richard W.M. Jones wrote:
 
  That's simply an idiotic thing to say.  What is the progress
  involved in adding a new BR to thousands of packages?
 
  Am I wrong assuming that it would be rather easy to add the BR to
  all packages semi-automatically previous to a mass rebuild so that
  people who want to get rid of gcc for their packages can easily
  remove it?
 
  What am I missing?
 
  What is missing is there is no quantifcation yet of the benefit.
  How much faster will builds be?
 
 
  Since you are asking I did 5 times:
 
  $ mock -r minimal --scrub all  time mock -r minimal --init
 
  Where the Total is DNF's output of package download speed and
  amount of downloaded data obtained from mock output. Other DNF
  information are shown on top of each section as well.
 
 
  == with current minimal build root, i.e. install @buildsys-build
 
  Install  164 Packages
 
  Total download size: 119 M
  Installed size: 445 M
 
  1)
  Total26 MB/s | 119
  MB 00:04 real0m51.364s
  user0m33.050s
  sys0m6.454s
 
  2)
  Total13 MB/s | 119
  MB 00:09 real0m51.479s
  user0m33.024s
  sys0m3.400s
 
  3)
  Total19 MB/s | 119
  MB 00:06 real0m52.373s
  user0m33.171s
  sys0m7.121s
 
  4)
  Total21 MB/s | 119
  MB 00:05 real0m54.213s
  user0m32.999s
  sys0m7.122s
 
  5)
  Total13 MB/s | 119
  MB 00:09 real0m56.093s
  user0m32.708s
  sys0m5.745s
 
 
  == without gcc, gcc-c++ and make, i.e. install rpm-build
  shadow-utils util-linux which
 
  Install  153 Packages
 
  Total download size: 81 M
  Installed size: 350 M
 
  1)
  Total14 MB/s |  81
  MB 00:05 real0m49.789s
  user0m28.926s
  sys0m6.864s
 
  2)
  Total17 MB/s |  81
  MB 00:04 real0m44.010s
  user0m28.965s
  sys0m3.434s
 
  3)
  Total18 MB/s |  81
  MB 00:04 real0m45.860s
  user0m29.032s
  sys0m5.730s
 
  4)
  Total19 MB/s |  81
  MB 00:04 real0m42.047s
  user0m28.944s
  sys0m3.099s
 
  5)
  Total14 MB/s |  81
  MB 00:05 real0m47.079s
  user0m29.839s
  sys0m3.978s
 
 
  == without gcc, gcc-c++, make and without perl (rhbz#1158860, note
  that the rpm packages are local in this case), i.e. install
  rpm-build shadow-utils util-linux which
 
  Install  124 Packages
 
  Total size: 69 M
  Total download size: 68 M
  Installed size: 310 M
 
  1)
  Total11 MB/s |  68
  MB 00:06 real0m44.725s
  user0m27.644s
  sys0m6.126s
 
  2)
  Total12 MB/s |  68
  MB 00:05 real0m46.253s
  user0m27.624s
  sys0m7.056s
 
  3)
  Total   6.5 MB/s |  68
  MB 00:10 real0m50.296s
  user0m27.567s
  sys0m6.064s
 
  4)
  Total12 MB/s |  68
  MB 00:05 real0m46.078s
  user0m27.581s
  sys0m7.698s
 
  5)
  Total16 MB/s |  68
  MB 00:04 real0m44.307s
  user0m27.830s
  sys0m5.293s
 
  == Only rpmbuild install install rpm-build shadow-utils
 
  Install  111 Packages
 
  Total size: 61 M
  Total download size: 60 M
  Installed size: 286 M
 
  1)
  Total20 MB/s |  60
  MB 00:03 real0m37.848s
  user0m26.158s
  sys0m2.676s
 
  2)
  Total13 MB/s |  60
  MB 00:04 real0m44.756s
  user0m26.442s
  sys0m5.185s
 
  3)
  Total16 MB/s |  60
  MB 00:03 real0m42.703s
  user0m26.118s
  sys0m3.174s
 
  4)
  Total13 MB/s |  60
  MB 00:04 real0m42.365s
  user0m26.439s
  sys0m4.193s
 
  5)
  Total   5.9 MB/s |  60
  MB 00:10 real0m48.940s
  user0m26.608s
  sys0m4.576s
 
  == Minimal root I can shell in, i.e. install bash shadow-utils
 
  Install  48 Packages
 
  Total download size: 31 M
  Installed size: 176 M
 
  1)
  Total   8.3 MB/s |  31
  MB 

Re: Remove gcc, gcc-c++ and make from minimal build root

[Miloslav Trmač]

 On Tue 13 Jan 2015 01:35:26 AM CET Kevin Kofler wrote:
  I actually think that cmake should be added to the minimal build root,
  instead of removing stuff. Almost all the packages I work on BuildRequire
  cmake (which also implies that they need make to build, and gcc-c++ is the
  typical implementation language).
 
 Yes good idea. I worked on Java packages. Let's put Maven in minimal
 buildroot. I am sure everyone will enjoy it.

And, you know, why not, exactly?  It’s not like the minimal buildroot is a 
cloud image where every megabyte is multiplied by hundreds of thousands of 
users and costs money.  Rather, the inevitable series of dozens of (yum install 
$library-devel) every developer does after installing a new OS is much more of 
a hassle than a few hundred megabytes of space used by default.

Yes, let’s agree (hah!) on good, stable, well-designed programming languages 
and libraries, and then install them by default in both the Workstation default 
install and the default build root.¹


 Seriously though what's a problem with listing your package's real build
 requires?

Life is too short to spend time micromanaging things that don’t actually matter 
for improving utility of the resulting applications to users.  Micromanaging 
the dependencies is not inherent to solving any application use case; it is 
purely accidental complexity / pointless hassle.

Lidem myšlení—strojům dřinu!²

(Incidentally, this would be much less of a problem if we had a way to add all 
the needed BuildRequires: automatically, and actually did that.  But the mere 
presence of these BuildRequires: in spec files would still be accidental 
complexity we are introducing into packaging—or alternatively, if we had a tool 
to detect this dependency then we wouldn’t need to explicitly list the 
dependency in the spec file.)
Mirek

¹ Now I am not saying that C/C++/make are that great that they would now 
warrant being chosen for to the default set but they are so prevalent that if 
there is any default set it has to include them.
² Let people think and machines toil.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Miloslav Trmač]

 Dne 13.1.2015 v 08:12 Ralf Corsepius napsal(a):
  On 01/13/2015 07:12 AM, Stanislav Ochotnicky wrote:
  On Tue 13 Jan 2015 01:35:26 AM CET Kevin Kofler wrote:
  Vít Ondruch wrote:
  I basically see several issues:
 
  1. The sheer amount of packages being affect.
snip data
By all accounts we are talking an order of thousands of packages, i.e. 
thousands of hours spent on adjusting the BuildRequires.  Conservatively, that 
could have been hundreds of user-visible bugs fixed; why is the user-invisible, 
and often developer-invisible, build speedup worth these hundreds of bugs?


  3. There likely are many tiny problems under the hood (esp. in
  packages primarily written in scripted languages), such as (yet
  unknown and hidden) conditionally built
  features/sub-components/sub-packages and conditional deps etc.
 
 Definitely, something will bite us. Nobody claims the opposite. Nobody
 says its piece of cake, lets do it for F22.

Looking at this in reverse, if we had the option to save thousands of hours and 
avoid unknown regressions by spending 20 seconds extra per build, why yes, we 
would very likely take it.  So shouldn’t we, consistently, not spend those 
thousands of hours and a risk of “definitely something will bite us” to save 20 
seconds per build?

Don’t get me wrong, speeding up builds is good in principle; but we have to 
consider the opportunity cost.  Couldn’t we find a way to completely automate 
away this decision so that the builds are fast and the packagers don’t have to 
care?
 Mirek
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Mikolaj Izdebski]

On 01/12/2015 02:08 PM, Vít Ondruch wrote:
 Dear Fedora developers,
 
 I'd like to collect some feedback about the $SUBJECT, i.e. making
 minimal build root really minimal, explicitly specifying build
 dependencies, etc.

I strongly support this idea.

In the past it probaby made sense to have GCC and friends in minimal
buildroot, but nowdays we have packages written in dozen of different
programming languages, which don't use GCC for compilation. It makes
perfect sense to drop C-related build tools from Koji @bulid group.

-- 
Mikolaj Izdebski
Software Engineer, Red Hat
IRC: mizdebsk
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Mikolaj Izdebski]

On 01/12/2015 02:30 PM, Marcin Juszkiewicz wrote:
 W dniu 12.01.2015 o 14:08, Vít Ondruch pisze:
 
 I'd like to collect some feedback about the $SUBJECT, i.e. making 
 minimal build root really minimal, explicitly specifying build 
 dependencies, etc.
 
 Are you aware that this means changing build dependencies of most of
 packages? Altering them just to add gcc, make etc.

There is no progress without change.

 Maybe also remove libc from minimal build root?

libc is not directly listed in @build. Among other things, it is
dependency of rpm, which will stay there.

-- 
Mikolaj Izdebski
Software Engineer, Red Hat
IRC: mizdebsk
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Bohuslav Kabrda]

- Original Message -
 Dear Fedora developers,
 
 I'd like to collect some feedback about the $SUBJECT, i.e. making
 minimal build root really minimal, explicitly specifying build
 dependencies, etc.
 
 The topic originally spring out in this [1] discussion on packaging
 list. Interesting FPC discussion on this topic can be find in this
 ticket [2] and meeting minutes from last FPC meeting [3].
 
 Thanks
 

+1 from Python land. Most Python modules don't need gcc and always installing 
it needlessly wastes bandwith and makes buildroot preparation in {mock,koji} 
slower.

 
 Vít
 
 
 
 [1]
 https://lists.fedoraproject.org/pipermail/packaging/2014-December/010398.html
 [2] https://fedorahosted.org/fpc/ticket/490
 [3]
 https://lists.fedoraproject.org/pipermail/packaging/2015-January/010424.html

-- 
Regards,
Slavek Kabrda
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Marcin Juszkiewicz]

W dniu 12.01.2015 o 14:08, Vít Ondruch pisze:

 I'd like to collect some feedback about the $SUBJECT, i.e. making 
 minimal build root really minimal, explicitly specifying build 
 dependencies, etc.

Are you aware that this means changing build dependencies of most of
packages? Altering them just to add gcc, make etc.

Maybe also remove libc from minimal build root?
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Dan Horák]

On Mon, 12 Jan 2015 14:30:49 +0100
Marcin Juszkiewicz mjuszkiew...@redhat.com wrote:

 W dniu 12.01.2015 o 14:08, Vít Ondruch pisze:
 
  I'd like to collect some feedback about the $SUBJECT, i.e. making 
  minimal build root really minimal, explicitly specifying build 
  dependencies, etc.
 
 Are you aware that this means changing build dependencies of most of
 packages? Altering them just to add gcc, make etc.

IIRC from the last mass-rebuilds, roughly 1/2 of packages are purely
noarch, the other 1/2 is producing binary rpm, which would include gcc
in most cases (maybe 95%). The question should be what is the ratio
when a BuildRequires should be part of the default build group.


Dan
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Vít Ondruch]

Dne 12.1.2015 v 14:30 Marcin Juszkiewicz napsal(a):
 W dniu 12.01.2015 o 14:08, Vít Ondruch pisze:

 I'd like to collect some feedback about the $SUBJECT, i.e. making 
 minimal build root really minimal, explicitly specifying build 
 dependencies, etc.
 Are you aware that this means changing build dependencies of most of
 packages? Altering them just to add gcc, make etc.

 Maybe also remove libc from minimal build root?

You may want to read my last comment in the FPC ticket, which tries to
address this a bit:

https://fedorahosted.org/fpc/ticket/490#comment:10

But some more info on this ...

If you want to have bash installed in the build root, these are the
packages which gets in:

Start: dnf install
Dependencies resolved.

 Package  Arch   Version  
Repository  Size

Installing:
 basesystem   noarch 10.0-10.fc21 
fedora 5.3 k
 bash x86_64 4.3.33-1.fc22
fedora 1.6 M
 fedora-release   noarch 22-0.10  
fedora  21 k
 fedora-repos noarch 22-0.2   
fedora  20 k
 fedora-repos-rawhide noarch 22-0.2   
fedora 9.0 k
 filesystem   x86_64 3.2-32.fc22  
fedora 1.1 M
 glibcx86_64 2.20.90-17.fc22  
fedora 3.5 M
 glibc-common x86_64 2.20.90-17.fc22  
fedora  12 M
 libgcc   x86_64 4.9.2-5.fc22 
fedora  87 k
 libselinux   x86_64 2.3-5.fc22   
fedora 149 k
 libsepol x86_64 2.3-4.fc22   
fedora 168 k
 libstdc++x86_64 4.9.2-5.fc22 
fedora 297 k
 ncurses-base noarch 5.9-17.20140906.fc22 
fedora  74 k
 ncurses-libs x86_64 5.9-17.20140906.fc22 
fedora 314 k
 nss-softokn-freebl   x86_64 3.17.3-1.fc22
fedora 207 k
 pcre x86_64 8.36-3.fc22  
fedora 483 k
 setupnoarch 2.9.4-1.fc22 
fedora 171 k
 tzdata   noarch 2014j-1.fc22 
fedora 420 k
 xz-libs  x86_64 5.2.0-1.fc22 
fedora  89 k

Transaction Summary

Install  19 Packages

Total download size: 20 M
Installed size: 143 M


You see that glibc is there, so it will be probably always installed.
But mock cannot successfully prepare the build root:

Complete!
Finish: dnf install
ERROR: Could not find useradd in chroot, maybe the install failed?


Adding shadow-utils and hence useradd command, these are the package
you'll get installed:



Start: dnf install
Dependencies resolved.

 PackageArch Version Repository
  
Size

Installing:
 audit-libs x86_64   2.4.1-1.fc22   
fedora93 k
 basesystem noarch   10.0-10.fc21   
fedora   5.3 k
 bash   x86_64   4.3.33-1.fc22  
fedora   1.6 M
 bzip2-libs x86_64   1.0.6-14.fc22  
fedora44 k
 ca-certificatesnoarch   2014.2.2-2.fc22
fedora   427 k
 chkconfig  x86_64   1.3.63-1.fc22  
fedora   176 k
 coreutils  x86_64   8.23-6.fc22
fedora   3.3 M
 crypto-policiesnoarch   20141124-2.gitd4aa178.fc22 
fedora25 k
 fedora-release noarch   22-0.10
fedora21 k
 fedora-repos   noarch   22-0.2 
fedora20 k
 fedora-repos-rawhide   noarch   22-0.2 
fedora   9.0 k
 filesystem x86_64   3.2-32.fc22
fedora   1.1 M
 gawk   x86_64   4.1.1-6.fc22   
fedora   1.0 M
 glibc  x86_64   2.20.90-17.fc22
fedora   3.5 M
 glibc-common   x86_64   2.20.90-17.fc22
fedora12 M
 gmpx86_64   1:6.0.0-8.fc22 
fedora   282 k
 grep   x86_64   2.21-2.fc22
fedora   364 k
 info   x86_64   5.2-7.fc22 
fedora   212 k
 keyutils-libs  x86_64   1.5.9-4.fc22   
fedora44 k
 krb5-libs  x86_64   1.13-4.fc22
fedora   838 k
 libacl 

Re: Remove gcc, gcc-c++ and make from minimal build root

[Peter Robinson]

On Tue, Jan 13, 2015 at 12:35 AM, Kevin Kofler kevin.kof...@chello.at wrote:
 Vít Ondruch wrote:
 I'd like to collect some feedback about the $SUBJECT, i.e. making
 minimal build root really minimal, explicitly specifying build
 dependencies, etc.

 -1, all the serious software requires gcc, gcc-c++ and make to build.

 I actually think that cmake should be added to the minimal build root,
 instead of removing stuff. Almost all the packages I work on BuildRequire
 cmake (which also implies that they need make to build, and gcc-c++ is the
 typical implementation language).

Of the 18K or so source packages a fairly low amount, I would doubt
it's in the double percentage figures, require cmake, so I don't see
why it's useful, just because it's what your subset of packages mostly
require it doesn't mean it's a generally wide requirement in the
distro itself!

Peter
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Kevin Kofler]

Vít Ondruch wrote:
 I'd like to collect some feedback about the $SUBJECT, i.e. making
 minimal build root really minimal, explicitly specifying build
 dependencies, etc.

-1, all the serious software requires gcc, gcc-c++ and make to build.

I actually think that cmake should be added to the minimal build root, 
instead of removing stuff. Almost all the packages I work on BuildRequire 
cmake (which also implies that they need make to build, and gcc-c++ is the 
typical implementation language).

Kevin Kofler

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Stanislav Ochotnicky]

On Tue 13 Jan 2015 01:35:26 AM CET Kevin Kofler wrote:
 Vít Ondruch wrote:
 I'd like to collect some feedback about the $SUBJECT, i.e. making
 minimal build root really minimal, explicitly specifying build
 dependencies, etc.

 -1, all the serious software requires gcc, gcc-c++ and make to build.

 I actually think that cmake should be added to the minimal build root,
 instead of removing stuff. Almost all the packages I work on BuildRequire
 cmake (which also implies that they need make to build, and gcc-c++ is the
 typical implementation language).

Yes good idea. I worked on Java packages. Let's put Maven in minimal
buildroot. I am sure everyone will enjoy it.

Sorry for the sarcasm. Couldn't resist :-)

Seriously though what's a problem with listing your package's real build
requires?

--
Stanislav Ochotnicky sochotni...@redhat.com
Business System Analyst, Hosted and Shared Services (HSS)

PGP: 7B087241
Red Hat Inc.   http://cz.redhat.com
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Ralf Corsepius]

On 01/13/2015 07:12 AM, Stanislav Ochotnicky wrote:

On Tue 13 Jan 2015 01:35:26 AM CET Kevin Kofler wrote:

Vít Ondruch wrote:

I'd like to collect some feedback about the $SUBJECT, i.e. making
minimal build root really minimal, explicitly specifying build
dependencies, etc.


-1, all the serious software requires gcc, gcc-c++ and make to build.

I actually think that cmake should be added to the minimal build root,
instead of removing stuff. Almost all the packages I work on BuildRequire
cmake (which also implies that they need make to build, and gcc-c++ is the
typical implementation language).


Yes good idea. I worked on Java packages. Let's put Maven in minimal
buildroot. I am sure everyone will enjoy it.

Sorry for the sarcasm. Couldn't resist :-)

Seriously though what's a problem with listing your package's real build
requires?


I basically see several issues:

1. The sheer amount of packages being affect.

Though I also think the ratio of scripted/compiled package has changed 
towards scripted languages, I do not think the number of compiled 
packages has decreased in absolute numbers and still is very high.


2. The semantics of such a change.

To me, they are not clear, because having gcc or gcc-c++ in a build 
root covers multiple use-cases.
For many packages, they just satisfy the need for an arbitrary c/c++ 
compiler, but actually do not mean a strict requirement to have 
gcc/g++. Similarly, they may exist packages which actually have 
dependencies on something which currently is being pulled in indirectly 
through gcc (say libgcc), while the package doesn't actually depend on a 
c-compiler.


3. There likely are many tiny problems under the hood (esp. in packages 
primarily written in scripted languages), such as (yet unknown and 
hidden) conditionally built features/sub-components/sub-packages and 
conditional deps etc.


I.e. though I am in favor of minimising the buildroots, I don't expect 
removing gcc/g++ will become an easy effort nor should it be taken 
light-heartedly.


Ralf



--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Remove gcc, gcc-c++ and make from minimal build root

[Michael Stahnke]

snip


  Vít Ondruch wrote:
  I'd like to collect some feedback about the $SUBJECT, i.e. making
  minimal build root really minimal, explicitly specifying build
  dependencies, etc.


+1

As somebody who works on software that sometimes get rebuilt by other
people not using koji or mock, I commonly have to explain what's in the
@build group or in the default mock roots. More explicit dependencies are
easier for everybody to track, even if just using rpmbuild -ba.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct