Re: [gentoo-dev] Re: debug/release builds extensions/clarification proposal
On Tue, Dec 2, 2008 at 11:19 PM, Maciej Mrozowski <[EMAIL PROTECTED]> wrote:
> On Tuesday 02 of December 2008 10:40:19 Alec Warner wrote:
>
>>
>> You asked, so the counter proposal is to *do nothing*.
>>
>> Ideas (even good ones) don't always get implemented.
>>
>> Sometimes that just isn't the direction the maintainers want to take
>> the project.
>> Sometimes it is harder to implement than most people realize.
>> Sometimes suggested implementations are just a hack and a bad idea all
>> around.
>>
>> I think starting with an implementation may have been a bad starting move.
>>
>> Start with what you want to accomplish:
>> - Get feedback on whether this is useful or not.
>> - Get feedback on other features that may be available.
>> - Get feedback on how some folks would accomplish this.
>>
>> "I want to be able to turn debug builds on or off on a per-package
>> basis. Debug builds entail both debugging symbols, split-debug, debug
>> CFLAGS and debug LDFLAGS."
>>
>> Is that a fair summary of your request?
>
> Yes, precisely. But forget about this proposal, as I stated already it's just
> a workaround for inability to set CFLAGS/LDFLAGS and those two FEATURES per-
> package basis in *official* way.
>
>> I am unsure how much you actually care about how each package manager
>> implements this feature (or if anyone implements it but portage, or
>> paludis, or whatever the majority of the KDE users are using).
>>
>> I'm also unsure how useful this is when say, some part of KDE links
>> against libfoo and KDE is built with debug symbols but libfoo is not.
>> Is that really useful? Are users actually asking for this proposed
>> feature or do you just think they want it? Do you have any data to
>> back up why someone should implement this feature (mailing list posts,
>> forums threads, etc..)
>
> No, and I'm afraid I cannot provide any single evidence that users actually
> need features like:
> - per package cflags/ldflags/features
> - per category use flags, accept_keywords, cflags
> - or tag clouds instead of hard coded categories
> - user-defined packages sets (official)
> - multiple portage configurations support to ease building binaries for
> several targets on a same host
> - dynamic libraries tracking for safe package upgrade or removal
> - real backwards dependencies
> - maybe git driven Portage
> - automatic kernel modules rebuilding
> - mysql split ebuilds
>
> Actually, I'm perfectly certain that users are way more interested in critical
> important aspects of their system like whether HOMEPAGE should be set in
> ebuilds or in metadata.xml :D
Sorry, I can't prioritize email threads ;)
>
> Please let me solve your little problem with HOMEPAGE for you...
> Package's homepage obviously may be, and actually is - ${PN}-${PV} specific.
> That being said it *would* needs to be specified either in every ebuild or as
> someone proposed - in metadata.xml in versioned/tagged way.
> And no matter how many searches you run - it may be easy to predict that due
> to lazyness (less probable) or just to avoid copy/paste (copy/paste is bad -
> everyone knows that) - some developers used to put HOMEPAGE in eclasses -
> because it may be used to put in postinst message for some reasons, that being
> said it needs to be in ebuild domain in current implementation.
> Mixing XML and bash (ebuild) in general isn't a bad idea but using bothe of
> them seems to be inconsistent - but some trade off needs to be paid sometimes.
> When duplicating HOMEPAGE is such a pain for developer (as he needs to type it
> all over again, I agree, it is pain, especially when one need to put some
> things only to please repoman), why not invest some time and develop tools
> that could make it easier - like meta-ebuilds (or ebuild generators) and
> ebuild templates? I've done something like this to autogenerate plasma applet
> live ebuilds from KDE playground on their SVN (it's not yet commited to
> overlay as eclass is not yet ready to fetch/unpack and build packages from
> this location and I haven't got time yet to patch it).
> If declaring HOMEPAGE in eclasses troubles you as you need BASH to process it
> properly (it may be pain for non-BASH search tools) and XML may be problematic
> to parse for bash tools - why not create such ebuild generator or 'compiler' -
> that could generate ebuild? Or for example as complete BASH script (no need
> for inherit anything) - and use eclasses ONLY like 'development library'.
> This way - every ebuild could be:
> - eclass-breakage free (overwriting eclasses don't take place so you are
> certain that user's emerge-problem is not him messing with eclasses - like
> mixing those from other overlays)
> - every defined variable is there (no need for 'inherit' lookup) - so that one
> can easily find HOMEPAGE= using every kind of tool (unless it is enclosed with
> some condition - why would anyone need to do that btw?)
> - much larger disk space requirements for Portage tree - but that could be
> compen
Re: [gentoo-dev] Re: debug/release builds extensions/clarification proposal
On Wed, 3 Dec 2008 08:19:18 +0100 Maciej Mrozowski <[EMAIL PROTECTED]> wrote: > No, and I'm afraid I cannot provide any single evidence that users > actually need features like: > - per package cflags/ldflags/features > - per category use flags, accept_keywords, cflags > - or tag clouds instead of hard coded categories > - user-defined packages sets (official) > - multiple portage configurations support to ease building binaries > for several targets on a same host > - dynamic libraries tracking for safe package upgrade or removal > - real backwards dependencies > - maybe git driven Portage > - automatic kernel modules rebuilding > - mysql split ebuilds Assuming that's a list of feature requests, you know that half of them are already available, right? (not counting the non-feature in there) Marius
Re: [gentoo-dev] Re: debug/release builds extensions/clarification proposal
On Tuesday 02 of December 2008 10:40:19 Alec Warner wrote:
>
> You asked, so the counter proposal is to *do nothing*.
>
> Ideas (even good ones) don't always get implemented.
>
> Sometimes that just isn't the direction the maintainers want to take
> the project.
> Sometimes it is harder to implement than most people realize.
> Sometimes suggested implementations are just a hack and a bad idea all
> around.
>
> I think starting with an implementation may have been a bad starting move.
>
> Start with what you want to accomplish:
> - Get feedback on whether this is useful or not.
> - Get feedback on other features that may be available.
> - Get feedback on how some folks would accomplish this.
>
> "I want to be able to turn debug builds on or off on a per-package
> basis. Debug builds entail both debugging symbols, split-debug, debug
> CFLAGS and debug LDFLAGS."
>
> Is that a fair summary of your request?
Yes, precisely. But forget about this proposal, as I stated already it's just
a workaround for inability to set CFLAGS/LDFLAGS and those two FEATURES per-
package basis in *official* way.
> I am unsure how much you actually care about how each package manager
> implements this feature (or if anyone implements it but portage, or
> paludis, or whatever the majority of the KDE users are using).
>
> I'm also unsure how useful this is when say, some part of KDE links
> against libfoo and KDE is built with debug symbols but libfoo is not.
> Is that really useful? Are users actually asking for this proposed
> feature or do you just think they want it? Do you have any data to
> back up why someone should implement this feature (mailing list posts,
> forums threads, etc..)
No, and I'm afraid I cannot provide any single evidence that users actually
need features like:
- per package cflags/ldflags/features
- per category use flags, accept_keywords, cflags
- or tag clouds instead of hard coded categories
- user-defined packages sets (official)
- multiple portage configurations support to ease building binaries for
several targets on a same host
- dynamic libraries tracking for safe package upgrade or removal
- real backwards dependencies
- maybe git driven Portage
- automatic kernel modules rebuilding
- mysql split ebuilds
Actually, I'm perfectly certain that users are way more interested in critical
important aspects of their system like whether HOMEPAGE should be set in
ebuilds or in metadata.xml :D
Please let me solve your little problem with HOMEPAGE for you...
Package's homepage obviously may be, and actually is - ${PN}-${PV} specific.
That being said it *would* needs to be specified either in every ebuild or as
someone proposed - in metadata.xml in versioned/tagged way.
And no matter how many searches you run - it may be easy to predict that due
to lazyness (less probable) or just to avoid copy/paste (copy/paste is bad -
everyone knows that) - some developers used to put HOMEPAGE in eclasses -
because it may be used to put in postinst message for some reasons, that being
said it needs to be in ebuild domain in current implementation.
Mixing XML and bash (ebuild) in general isn't a bad idea but using bothe of
them seems to be inconsistent - but some trade off needs to be paid sometimes.
When duplicating HOMEPAGE is such a pain for developer (as he needs to type it
all over again, I agree, it is pain, especially when one need to put some
things only to please repoman), why not invest some time and develop tools
that could make it easier - like meta-ebuilds (or ebuild generators) and
ebuild templates? I've done something like this to autogenerate plasma applet
live ebuilds from KDE playground on their SVN (it's not yet commited to
overlay as eclass is not yet ready to fetch/unpack and build packages from
this location and I haven't got time yet to patch it).
If declaring HOMEPAGE in eclasses troubles you as you need BASH to process it
properly (it may be pain for non-BASH search tools) and XML may be problematic
to parse for bash tools - why not create such ebuild generator or 'compiler' -
that could generate ebuild? Or for example as complete BASH script (no need
for inherit anything) - and use eclasses ONLY like 'development library'.
This way - every ebuild could be:
- eclass-breakage free (overwriting eclasses don't take place so you are
certain that user's emerge-problem is not him messing with eclasses - like
mixing those from other overlays)
- every defined variable is there (no need for 'inherit' lookup) - so that one
can easily find HOMEPAGE= using every kind of tool (unless it is enclosed with
some condition - why would anyone need to do that btw?)
- much larger disk space requirements for Portage tree - but that could be
compensated by for example gzipping every ebuild.
Of course every problems with dichotomy ebuild vs metadata could be solved by
some new Portage backend - better suited for queries and storage (maybe some
relational database).
But so far - the
Re: [gentoo-dev] Re: debug/release builds extensions/clarification proposal
On Mon, Dec 1, 2008 at 3:14 PM, Maciej Mrozowski <[EMAIL PROTECTED]> wrote:
> On Monday 01 of December 2008 22:51:57 Ciaran McCreesh wrote:
>
>> Experience, manpower, the ability to try out potential enhancements
>> rapidly, a long track record of getting it right and the growing
>> recognition that most people doing package manager work for Gentoo
>> aren't doing it with Portage.
>
> While of course I agree that any input from 'outside' is welcome and valuable,
> yet to get things done, in my opinion the final decision should not be blocked
> by from any alternative package manager and some policies should be enforced.
>
> But on topic, what's a counter proposal for my idea then?
You asked, so the counter proposal is to *do nothing*.
Ideas (even good ones) don't always get implemented.
Sometimes that just isn't the direction the maintainers want to take
the project.
Sometimes it is harder to implement than most people realize.
Sometimes suggested implementations are just a hack and a bad idea all around.
I think starting with an implementation may have been a bad starting move.
Start with what you want to accomplish:
- Get feedback on whether this is useful or not.
- Get feedback on other features that may be available.
- Get feedback on how some folks would accomplish this.
"I want to be able to turn debug builds on or off on a per-package
basis. Debug builds entail both debugging symbols, split-debug, debug
CFLAGS and debug LDFLAGS."
Is that a fair summary of your request?
I am unsure how much you actually care about how each package manager
implements this feature (or if anyone implements it but portage, or
paludis, or whatever the majority of the KDE users are using).
I'm also unsure how useful this is when say, some part of KDE links
against libfoo and KDE is built with debug symbols but libfoo is not.
Is that really useful? Are users actually asking for this proposed
feature or do you just think they want it? Do you have any data to
back up why someone should implement this feature (mailing list posts,
forums threads, etc..)
Certainly for portage per-package features are possible with a minor
patch (to read the custom settings from your config and to inject the
FEATURES variables into the per-package config when necessary). The
problem that has been stated in the past is that FEATURES were not
designed to be used in that manner (per-package).
We could design an separate system that let you define per-package
'things' and use these 'things' to trigger debug builds (completely
outside of FEATURES, leaving them alone). FEATURES were in fact
specific features of portage that you want 'on' or 'off'
(metadata-transfer, parallel-fetch, userfetch, unmerge-orphans,
etc...) These are examples of things you would not turn off
per-ebuild.
But the question is always 'is it really worth it' and can you get
someone to do it.
Sometimes, doing nothing is better than doing something badly.
-Alec
> Quick search in archives gave me some results I don't particularly like, like
> the idea with /etc/portage/packages.cflags and /etc/portage/package.env, and
> they have been dropped for similar reasons - as the former needs special
> parsing instead just sourcing the script (the problem is that someone needs to
> implement this - this is usually the problem, especially in pure volunteer
> projects like Gentoo), the latter looks a bit messy to me. /etc/portage/env
> would be the best approach when made officially supported (recently it looks
> like /etc/portage/env is sourced multiple times and that should be fixed, for
> convenience, just in case user wants to put:
> CFLAGS="-O0 -ggdb"
> CXXFLAGS="${CFLAGS}"
> FEATURES="${FEATURES} nostrip"
> (or even USE="${USE} debug")
> actually /etc/portage/env could easily replace package.keywords and
> package.use as well and introduce replacement for meybe-proposed-sometime
> package.features - I wonder whether it's been discussed already.
Not without causing a bunch of pain in figuring out the inheriting
order of stack USE variables.
>
> --
> regards
> MM
>
Re: [gentoo-dev] Re: debug/release builds extensions/clarification proposal
On Monday 01 of December 2008 22:51:57 Ciaran McCreesh wrote:
> Experience, manpower, the ability to try out potential enhancements
> rapidly, a long track record of getting it right and the growing
> recognition that most people doing package manager work for Gentoo
> aren't doing it with Portage.
While of course I agree that any input from 'outside' is welcome and valuable,
yet to get things done, in my opinion the final decision should not be blocked
by from any alternative package manager and some policies should be enforced.
But on topic, what's a counter proposal for my idea then?
Quick search in archives gave me some results I don't particularly like, like
the idea with /etc/portage/packages.cflags and /etc/portage/package.env, and
they have been dropped for similar reasons - as the former needs special
parsing instead just sourcing the script (the problem is that someone needs to
implement this - this is usually the problem, especially in pure volunteer
projects like Gentoo), the latter looks a bit messy to me. /etc/portage/env
would be the best approach when made officially supported (recently it looks
like /etc/portage/env is sourced multiple times and that should be fixed, for
convenience, just in case user wants to put:
CFLAGS="-O0 -ggdb"
CXXFLAGS="${CFLAGS}"
FEATURES="${FEATURES} nostrip"
(or even USE="${USE} debug")
actually /etc/portage/env could easily replace package.keywords and
package.use as well and introduce replacement for meybe-proposed-sometime
package.features - I wonder whether it's been discussed already.
--
regards
MM
signature.asc
Description: This is a digitally signed message part.
Re: [gentoo-dev] Re: debug/release builds extensions/clarification proposal
On Mon, 1 Dec 2008 22:46:18 +0100 Maciej Mrozowski <[EMAIL PROTECTED]> wrote: > That I found interesting - what does any 3rd party package manager to > do with setting policies and enhancements regarding official Gentoo > package manager? Experience, manpower, the ability to try out potential enhancements rapidly, a long track record of getting it right and the growing recognition that most people doing package manager work for Gentoo aren't doing it with Portage. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: debug/release builds extensions/clarification proposal
On Monday 01 of December 2008 08:04:04 Duncan wrote: > (Of > course, if it's the latter, it will need to be an official GLEP, and > you'll have three separate package managers and their developers to push > the proposal thru to at least to general agreement, or the council will > almost certainly reject the GLEP, if it gets even that far.) That I found interesting - what does any 3rd party package manager to do with setting policies and enhancements regarding official Gentoo package manager? Have you ever heard of "liberum veto"? But that's an off topic of course. -- regards MM signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Re: debug/release builds extensions/clarification proposal
On Monday 01 of December 2008 09:36:12 Diego 'Flameeyes' Pettenò wrote:
> > - USE=debug is useless when CFLAGS/LDFLAGS or FEATURES are not
> > appropriate
> What are you saying here? I'm afraid you're mistaken here.
The point is to look at this from users' (well, a bit) point of view -
USE=debug variable is ambiguous in it's meaning. While it enables only
codepaths (asserts, #ifdefs and similar) it suggests (by name and for some
packages not only suggests) enabling debug symbols.
And policy is to enforce CFLAGS from make.conf and wipe out every package-
defined flags as far as I know.
> For the most part, USE=debug means "enable debug code paths", which for
> lots of projects simply means "enable assertions"; there are packages
> that take this as "enable debug symbols too" but I don't think that's
> very valid since users might want debug code paths but not symbols and
> vice-versa (I indeed have debug symbols bug no debug codepaths enabled).
That's correct, the problem is - Gentoo does not provide officially supported
mechanism of enabling both or just debug symbols per package basis - it
doesn't even provide any supported/documented mechanism for per package
CFLAGS, FEATURES and similar.
If /etc/portage/env hack/feature could be made official (for CFLAGS,LDFLAGS
and bash-domain FEATURES) - it could address this issue good enough, because
with proper smart combination of symlinks/files the "ultimate configuration"
power would be delivered, not just "cleaning/workaround" I am actually
proposing. Per package debug/release/profile/or_any_other configuration is
what I would pursue, and in my proposal I used USE=debug as existing and
supported way of achieving this.
While I don't like hack @pve uses (I prefer portage/env as more convenient
way), his idea about emerge --info seems interesting.
> - -ggdb *does not have any runtime performance hit*; neither in
Yes, I'm well aware of that, though it increases disk space requirements a bit
as it's applied to all libs/bins.
> - -O0 is not always a good idea; beside bugs in packages concealed by
>-O1+ [1]
[1] is a pathology and should be fought against, -O1+ may leave frame stack
useless for debugging due to inline optimizations in some places (especially
debugging inline class implementations is limited, which affects Qt/KDE) -
besides - I may not stated it clear - those default values would be defined in
the very same make.conf, so it could be:
CHOST="x86_64-pc-linux-gnu"
CFLAGS="-march=nocona -O2 -pipe -msse3 -ftree-vectorize"
CXXFLAGS="${CFLAGS}"
CFLAGS_DEBUG="-O2 -ggdb"
Yet, I still cannot think of this proposal other way like of dirty workaround
for the problem, that doesn't really exist (well, at least for developers, who
have meta-distribution and ultimate freedom for user in mind). For the users
the problem is real, of course it's usually a consequence of either not being
aware of those mechanisms or as a result of ambiguous semantics of USE=debug.
And what about pushing some bash-domain FEATURES to USE flags? Like nostrip,
splitdebug? I guess being able to set it per package is important.
--
regards
MM
signature.asc
Description: This is a digitally signed message part.
Re: [gentoo-dev] Re: debug/release builds extensions/clarification proposal
On Monday 01 of December 2008 08:04:04 Duncan wrote:
Well, so far it's not GLEP, just an idea thrown to brainstorm.
> As such, neither /etc/portage/env nor eclasses can effectively deal with
> FEATURES in general, tho there are a few specific exceptions that do
> happen to be implemented at the bash level.
Those exceptions are nostrip and splitdebug at least, besides I intend to keep
it bash (or ebuild) level only - to preserve simplicity and yet functionality.
FEATURES_DEBUG was a clean and convenient approach of me being unaware of
FEATURES internals - thanks for clarification. FEATURES little inconsistency
problem needs to be addressed. The goal is to have only one, determined and
always working way of "not-stripping" symbols. Of course it can be easily
handled in eclass by something like this:
if use debug; then
FEATURES=${FEATURES//splitdebug//}
FEATURES=${FEATURES//nostrip//}
FEATURES="${FEATURES} ${PREFERRED_NOSTRIP_METHOD}"
Dzwon tanio do wszystkich!
Sprawdz >> http://link.interia.pl/f1fa7
[gentoo-dev] Re: debug/release builds extensions/clarification proposal
Maciej Mrozowski <[EMAIL PROTECTED]> writes: > - USE=debug is useless when CFLAGS/LDFLAGS or FEATURES are not appropriate What are you saying here? I'm afraid you're mistaken here. For the most part, USE=debug means "enable debug code paths", which for lots of projects simply means "enable assertions"; there are packages that take this as "enable debug symbols too" but I don't think that's very valid since users might want debug code paths but not symbols and vice-versa (I indeed have debug symbols bug no debug codepaths enabled). Now just to make sure the common misconceptions don't hit again: - -ggdb *does not have any runtime performance hit*; neither in execution time nor in memory usage; the debug sections are not mapped into memory at all; this is true for both non-stripped and split executables; - -O0 is not always a good idea; beside bugs in packages concealed by -O1+ [1], there are some further points: missing registers on x86 causes build failures, and if ( 0 ) cases are not optimised away, resulting in stuff like FFmpeg not to link properly since undefined references are not pruned away; this means that using -O0 unconditionally for any package for debug is not really an option; [1] http://blog.flameeyes.eu/2008/09/02/testing-the-corner-cases -- Diego "Flameeyes" Pettenò http://blog.flameeyes.eu/ pgpJ5ioEeo3pE.pgp Description: PGP signature
[gentoo-dev] Re: debug/release builds extensions/clarification proposal
Maciej Mrozowski <[EMAIL PROTECTED]> posted [EMAIL PROTECTED], excerpted below, on Mon, 01 Dec 2008 06:16:07 +0100: > Implementation: > Implementation would be provided by build system eclasses [snip] > - replace FEATURES with FEATURES_DEBUG FEATURES are package-manager implemented, above the bash level where eclasses are parsed and executed, thus for portage, at the python level. As such, neither /etc/portage/env nor eclasses can effectively deal with FEATURES in general, tho there are a few specific exceptions that do happen to be implemented at the bash level. Thus, your GLEP (Gentoo Linux Enhancement Proposal) needs to specifically address this problem, either stating that this FEATURE can be implemented 100% at the bash/eclass level with details, or omitting/changing the FEATURE portion so it will work at the bash/eclass level, or outlining specifically what the package manager implementation must be. (Of course, if it's the latter, it will need to be an official GLEP, and you'll have three separate package managers and their developers to push the proposal thru to at least to general agreement, or the council will almost certainly reject the GLEP, if it gets even that far.) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
