Re: private so provides filtering...
On Mon, May 25, 2009 at 9:08 AM, Stepan Kasal ska...@redhat.com wrote: But it would still be better if the macro contained a list of regular expressions that would get filtered by one instance of grep, or even by the find itself. So there is the need for the general mechanism. But we might decide to push the general mechanism later, as a second iteration. Would you be more comfortable if I ran this by the rpm maintainers first, or the fedora-packaging list? Or do you have an alternate set of macros we should consider? -Chris -- Chris Weyl Ex astris, scientia -- Fedora Extras Perl SIG http://www.fedoraproject.org/wiki/Extras/SIGs/Perl Fedora-perl-devel-list mailing list Fedora-perl-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-perl-devel-list
Re: private so provides filtering...
Hello Chris,
http://fedorapeople.org/~cweyl/macros.perlhttp://fedorapeople.org/%7Ecweyl/macros.perl
[snip]
If the macros look sane, I'll open a RFE bug against the perl package to
ask that it be bundled and delivered as /etc/rpm/macros.perl.
first, I'd like to join the wonderful chorus, both for starting
this topic and for going the direction towards more general solution.
But I think that the requirements are generally useful, they
shouldn't be limited to perl packaging.
The requirement to skip %{_docdir} seems safe for all packages.
Skipping perl_vendorarch/.../*.so doesn't sound much general, but I
believe we could change that rule: auto-provides should count only
lib*.so libraries, not all *.so.
I think we should really push these changes to the rpm package itself
(or at least to redhat-rpm-config).
I'm not much happy with the actual implementation in macros.perl:
We should not build a pipe of grep commands; one grep in a pipe is
enough. Moreover, the %{_docdir} subtree should be skipped by find
in the first place (-prune is your friend).
To sum up, I'd like to push the actual filters to rpm build defaults,
but I'm not so sure about pushing the general mechanism to amend the
filter macros.
Cheers,
Stepan
--
Fedora Extras Perl SIG
http://www.fedoraproject.org/wiki/Extras/SIGs/Perl
Fedora-perl-devel-list mailing list
Fedora-perl-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-perl-devel-list
Re: private so provides filtering...
On Mon, May 25, 2009 at 8:49 AM, Stepan Kasal ska...@redhat.com wrote:
Hello Chris,
http://fedorapeople.org/~cweyl/macros.perlhttp://fedorapeople.org/%7Ecweyl/macros.perl
http://fedorapeople.org/%7Ecweyl/macros.perl
[snip]
If the macros look sane, I'll open a RFE bug against the perl
package to
ask that it be bundled and delivered as /etc/rpm/macros.perl.
first, I'd like to join the wonderful chorus, both for starting
this topic and for going the direction towards more general solution.
But I think that the requirements are generally useful, they
shouldn't be limited to perl packaging.
I definitely agree. However, there are a couple reasons I'd suggest we
start here:
1) start simple, and prove that there's a need/demand for it.. Heck, we
might find a change we need to make; this is much easier to do if we start
out owning it.
2) the maintainers of the rpm package has historically been resistant to
putting in place any sort of default filtering, or making the filtering
process easier. (see any one of a half-dozen or so bugs, everything from
the results of the prov/dep scripts[1] to not scanning private libs for
prov[2], to a more aggressive file causing bits in _docdir to be
scanned[3], to not wanting to alter _docdir scanning[4]... and those are
just the ones I've filed). If we're going to get these macros into the base
rpm package, which I agree is a good direction to go, I believe we're going
to need to have a pretty solid base using them first.
Also, just because we'd be delivering them in a macros.perl file doesn't
mean that their usage would be limited to just perl packages: perl is in
every buildroot and on every system, AFAIK. Those macros will always be
available to any system; and for the most part should be invoked
conditionally anyways.
The requirement to skip %{_docdir} seems safe for all packages.
Yep. And would certainly make life easier for a bunch of people :)
Skipping perl_vendorarch/.../*.so doesn't sound much general, but I
believe we could change that rule: auto-provides should count only
lib*.so libraries, not all *.so.
A good idea, but one that the rpm maintainers have historically resisted. I
don't think we should wait to solve our problems on this; if anything it
sounds like an effort that could be carried on in parallel.
For our packages, we should skip any provides of the .so's in
vendorarch/archlib as the perl dependency resolution will never make use of
these... And nothing else on the system should, either.
I think we should really push these changes to the rpm package itself
(or at least to redhat-rpm-config).
Agreed, but see above :) Let's solve our problems, then get them in core
rpm... Given the historical trend of rpm resisting any efforts along these
lines, I think we have to do it this way to be successful.
I'm not much happy with the actual implementation in macros.perl:
We should not build a pipe of grep commands; one grep in a pipe is
enough. Moreover, the %{_docdir} subtree should be skipped by find
in the first place (-prune is your friend).
Yeah, I'd like that too. There are a couple factors in favor of the current
proposed implementation:
It works.
rpm macros are a royal pain, and keeping it simple is best for our
collective sanity.
And... so we have a couple extra forks at the end of an rpm build process.
Big deal :) It's not like this is either time-critical or is going to
execute more than once per package build; the simplicity gained from this
approach far outweighs any negative from the additional processes.
For instance, in the current implementation I could easily add additional
filtering while still using the base default macro, without having to
worry about making sure everything fits under one grep... The net result of
each filter macro invocation is cumulative, simple, and easy:
%filter_from_requires /^perl(DBD::Pg)$/d
%perl_default_filter
What if we decided at some point to allow perl to be used to do the
filtering? Or needed extra options passed to grep? (as we do for the PCRE
that is the perl_archlib statement) RPM macros and filtering are painful
enough. Let's just use a set of simple and effective macros that Just
Work and we don't have to spend a ton of brain cycles on :)
That being said, I'm open to looking at any alternative set of macros :)
As to _docdir being skipped outright by base rpm: I don't think that's going
to happen, ever. I think it's the right thing to do, but I don't get to
make those calls and it's been repeatedly shown that it's not going to
happen.
To sum up, I'd like to push the actual filters to rpm build defaults,
but I'm not so sure about pushing the general mechanism to amend the
filter macros.
Hey, we have to start somewhere :) Every interaction I've seen or read with
the rpm maintainers leads me to believe that they're not going to want this
in base rpm without a hugely compelling reason first... we need it and
the current methodology is
Re: private so provides filtering...
On Sun, May 24, 2009 at 3:03 AM, Chris Weyl cw...@alumni.drew.edu wrote: On Sat, May 23, 2009 at 5:57 PM, Chris Weyl cw...@alumni.drew.edu wrote: So, in today's rawhide there are still 184 perl-* packages that provide a .so they shouldn't. If there aren't any screams, sometime in the not too distant future I'm going to loop over them, add the filtering code we hashed out earlier, and kick builds off in rawhide. ...or, of course, someone has a better / cleaner idea :) I hate to see that much copy/pasted code - wouldn't it be better to have a more centralized solution. Maybe an /etc/rpm/macros.perl with macros for filtering private .so and unwanted perl(...) provides/requires? At least until someone gets the time to implement a better requires/provides facility in rpm (or redhat-rpm-config) itself - at which point, these workarounds only need to be undone in one place. -- Iain. -- Fedora Extras Perl SIG http://www.fedoraproject.org/wiki/Extras/SIGs/Perl Fedora-perl-devel-list mailing list Fedora-perl-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-perl-devel-list
Re: private so provides filtering...
On Sat, May 23, 2009 at 11:13 PM, Iain Arnell iarn...@gmail.com wrote:
On Sun, May 24, 2009 at 3:03 AM, Chris Weyl cw...@alumni.drew.edu wrote:
On Sat, May 23, 2009 at 5:57 PM, Chris Weyl cw...@alumni.drew.edu
wrote:
So, in today's rawhide there are still 184 perl-* packages that provide
a
.so they shouldn't. If there aren't any screams, sometime in the not
too
distant future I'm going to loop over them, add the filtering code we
hashed
out earlier, and kick builds off in rawhide.
...or, of course, someone has a better / cleaner idea :)
I hate to see that much copy/pasted code - wouldn't it be better to
have a more centralized solution. Maybe an /etc/rpm/macros.perl with
macros for filtering private .so and unwanted perl(...)
provides/requires? At least until someone gets the time to implement a
better requires/provides facility in rpm (or redhat-rpm-config) itself
- at which point, these workarounds only need to be undone in one
place.
Heh. I actually sent that last note, then started going down the macro road
myself... Initially I looked at just doing something specific to this, but
broke it out and wrote something a touch more granular:
http://fedorapeople.org/~cweyl/macros.perl
With that in place, it's possible to stick a %{?perl_default_filter}, say
before %description, causing all private .so's to not be scanned by the
requires scripts... It also prevents anything under %{_docdir} from being
scanned for either prov or req. We can also filter things both at the file
level (preventing them from being scanned) or at the generated req/dep
level.
If the macros look sane, I'll open a RFE bug against the perl package to ask
that it be bundled and delivered as /etc/rpm/macros.perl.
While this is aimed at solving our issues, it should make dealing with wonky
auto req/provs much easier. :)
-Chris
--
Chris Weyl
Ex astris, scientia
--
Fedora Extras Perl SIG
http://www.fedoraproject.org/wiki/Extras/SIGs/Perl
Fedora-perl-devel-list mailing list
Fedora-perl-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-perl-devel-list
Re: private so provides filtering...
On Sun, May 24, 2009 at 9:39 AM, Chris Weyl cw...@alumni.drew.edu wrote: http://fedorapeople.org/~cweyl/macros.perl [snip] If the macros look sane, I'll open a RFE bug against the perl package to ask that it be bundled and delivered as /etc/rpm/macros.perl. Wonderful. Certainly looks sane and appears to be working as expected. Would also be sensible to update the specfile templates in rpmdevtools and cpanspec (and others?) to include this by default. And presumably packages relying on this would also need something along the lines of BuildRequires: config(perl). -- Iain. -- Fedora Extras Perl SIG http://www.fedoraproject.org/wiki/Extras/SIGs/Perl Fedora-perl-devel-list mailing list Fedora-perl-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-perl-devel-list
Re: private so provides filtering...
On Sun, May 24, 2009 at 5:41 AM, Iain Arnell iarn...@gmail.com wrote: On Sun, May 24, 2009 at 9:39 AM, Chris Weyl cw...@alumni.drew.edu wrote: http://fedorapeople.org/~cweyl/macros.perlhttp://fedorapeople.org/%7Ecweyl/macros.perl [snip] If the macros look sane, I'll open a RFE bug against the perl package to ask that it be bundled and delivered as /etc/rpm/macros.perl. Wonderful. Certainly looks sane and appears to be working as expected. Good. RPM macros always hurt my brain a little bit :) Would also be sensible to update the specfile templates in rpmdevtools and cpanspec (and others?) to include this by default. And presumably packages relying on this would also need something along the lines of BuildRequires: config(perl). A couple things work in our favour here. If we use the core perl package to deliver the macros to /etc/rpm/macros.perl, then for any other perl package we build we're covered. If a non-perl package wants to use it, then they're covered as well: either by requiring perl explicitly or simply relying on perl always being in the buildroot (which it is). So, once the new macro file is out there we'll be able to use them anywhere without needing to explicitly require anything new... Though a conditional invokation is probably best for a while. If other packages start to use them widescale, that will give us leverage to have them included in the rpm package. -Chris -- Chris Weyl Ex astris, scientia -- Fedora Extras Perl SIG http://www.fedoraproject.org/wiki/Extras/SIGs/Perl Fedora-perl-devel-list mailing list Fedora-perl-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-perl-devel-list
Re: private so provides filtering...
On Sun, May 24, 2009 at 12:29 PM, Chris Weyl cw...@alumni.drew.edu wrote:
On Sun, May 24, 2009 at 5:41 AM, Iain Arnell iarn...@gmail.com wrote:
On Sun, May 24, 2009 at 9:39 AM, Chris Weyl cw...@alumni.drew.edu
wrote:
http://fedorapeople.org/~cweyl/macros.perlhttp://fedorapeople.org/%7Ecweyl/macros.perl
[snip]
If the macros look sane, I'll open a RFE bug against the perl package to
ask
that it be bundled and delivered as /etc/rpm/macros.perl.
Wonderful. Certainly looks sane and appears to be working as expected.
Good. RPM macros always hurt my brain a little bit :)
A couple updates: %perl_default_filter no longer filters out
%{perl_archlib}/CORE/libperl.so; %filter_requires/provides_in now takes a -P
flag, to tell grep to use PCRE's; sed now properly appears as %{__sed}.
-Chris
--
Chris Weyl
Ex astris, scientia
--
Fedora Extras Perl SIG
http://www.fedoraproject.org/wiki/Extras/SIGs/Perl
Fedora-perl-devel-list mailing list
Fedora-perl-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-perl-devel-list
private so provides filtering...
So, in today's rawhide there are still 184 perl-* packages that provide a .so they shouldn't. If there aren't any screams, sometime in the not too distant future I'm going to loop over them, add the filtering code we hashed out earlier, and kick builds off in rawhide. http://fedorapeople.org/~cweyl/private_libs If you'd rather I didn't touch your package for whatever reason, or want to do it yourself, either let me know or add the filter and rebuild. :-) -Chris -- Chris Weyl Ex astris, scientia -- Fedora Extras Perl SIG http://www.fedoraproject.org/wiki/Extras/SIGs/Perl Fedora-perl-devel-list mailing list Fedora-perl-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-perl-devel-list
Re: private so provides filtering...
On Sat, May 23, 2009 at 5:57 PM, Chris Weyl cw...@alumni.drew.edu wrote: So, in today's rawhide there are still 184 perl-* packages that provide a .so they shouldn't. If there aren't any screams, sometime in the not too distant future I'm going to loop over them, add the filtering code we hashed out earlier, and kick builds off in rawhide. ...or, of course, someone has a better / cleaner idea :) -Chris Chris Weyl Ex astris, scientia -- Fedora Extras Perl SIG http://www.fedoraproject.org/wiki/Extras/SIGs/Perl Fedora-perl-devel-list mailing list Fedora-perl-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-perl-devel-list
