Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Monday 20 April 2015 12:46 PM, David Kalnischkies wrote: That can be very quickly quite a set of packages. apt ~23, apititude ~40, mpv (similar to mplayer) ~159, kate (KDEs notepad) ~465. [0] That can be tuned by excluding non-libraries, but that has its own drawbacks (private libraries shared between a very closely related set of packages for example), aka: For a quickshot direct dependencies are probably enough (personal observation; the times I needed debug symbols for non-direct dependencies are far and in between, but maybe I am just lucky). If you wanna go fullcircle, its probably better to analyse a core-dump for which symbols are needed exactly instead of getting everything. I think Ubuntu has a tool dubbed apport-retrace (Debian has it in experimental only) which is supposed to do that (but I just remember hearing the name in this context, nothing more) Just FYI. Apport is included in this year's GSoC, for Debian. There are a bunch of Debian specific features we'd like to see. Once those are done, we should be in a better position to propose apport for testing/unstable. And apport will be one of the prime consumers of these debug symbols. So thank you for reviving on this subject. -- Given the large number of mailing lists I follow, I request you to CC me in replies for quicker response signature.asc Description: OpenPGP digital signature
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On 2015-05-03 18:58, Guillem Jover wrote: Hi! On Tue, 2015-04-07 at 22:11:18 +0200, Niels Thykier wrote: A) Use .deb (i.e. the regular extension) with a new section. Is there any problem with using the existing debug section? Or is the different section used to distinguish that these are autogenerated perhaps? I picked debugsym to ensure it distinguished the automatically generated debug debs from hand-crafted ones while testing it. I am personally slightly in favour of keeping this as an (extra) discriminator. That said, I will not object to using the normal debug section. B) Use .ddeb (i.e. with an extra d). What where the motivations for using a different extension? I have heard of/can think of two reasons: * it was to have a trivial discriminator prior to unpacking / extracting information from the deb. * backwards compatibility with Ubuntu's setup. I can see the motivation for .udeb:s as they need to live in a different namespace, but that does not seem to be the case for debug debs. Please keep in mind that there /is/ a desire to keep ddebs trivially distinguishable from regular debs. Among other things, to facilitate putting ddebs in a separate part of the mirror to avoid inflating the size of the metadata on the mirror (for users not using ddebs). Assuming that any issue with debug .deb:s is fixed in the tools, is there any remaining reason to use .ddeb:s? To my knowledge: No - dpkg-genchanges and dak are the only known tools blocking the use of .deb. To be honest, I am not sure about dak, as I have not tried uploading with an out-of-band deb. That said, dak will want to know about them to avoid an explosion in the NEW queue. On 2015-04-07 21:10, Niels Thykier wrote: Both have their own advantages and disadvantages. In particular: A) means that almost every existing tool will handle the debug debs like a regular deb (which it is) and will generally work perfectly out of the box. - There are a couple of exceptions, but we are limited to something like lintian and dpkg-genchanges. I'd happily look into making dpkg-genchanges allow this. Thanks. I tried dpkg from git, which now allows them with only a warning. I would certainly be interested in having a (long-term) solution that did not warn about these. - There will be tools that might want to handle them differently. Programs like dak and reprepro might want to (support) put(ting) them in a different part of their repositories. They could already do that by keying on the Section, no? Otherwise the filename is also unique enough to be keyed on («*-dbgsym_*»)? That is my understanding as well. Note that Ubuntu have a few manually generated *-dbgsym_* packages (e.g. for the linux kernel). It is my understanding that these are in spirit intended to be considered regular ddebs. Accordingly, there ought to be no issue in filtering based on that file name pattern. [...] From my point of view, I am not strongly attached to one solution over the other: * I am slightly preferring A), but I am ready to implement either solution in the tools, I maintain. * The difference for debhelper is a single d and a section name. * The change for lintian is larger, but B) is the heavy solution and I already got a mostly working patch for that. Barring any strong reasoning behind B) above, I pretty much prefer A). Thanks, Guillem Great. :) To my knowledge, A) is also the solution that the FTP masters seemed to prefer. Thanks, ~Niels -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/55545904.2070...@thykier.net
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On 2015-04-19 19:10, David Kalnischkies wrote: On Sat, Apr 04, 2015 at 10:54:09AM +0200, Niels Thykier wrote: The resulting debs are installable with dpkg -i ( \o/ ). I have not tried anything fancy like setting up a local APT mirror and tried to convince APT do install it. I did and apt works with ddeb just fine, meaning it can happily print infos about them, download them and install them with dpkg. There are two exceptions as far as I can see: [... snip minor issues with .ddeb support in various APT tools ...] For reference, it seems like we will be picking the .deb route. So even these (minor) issues are unlikely to be a problem. So, super-cow approves (d)debs. \o/ (In fact, apt-dbg never became a thing as automatic ddebs were always very soon now in existence every time it came up. This time please…) I certainly intend to follow this all the way. And it especially approves the debhelper branch name. ;) I could imagine. : [...] btw: Is it planed to drop them into their own repository/component or are we gone blow up our regular Packages files with them? (you might guess from the wording which way I would prefer). Best regards David Kalnischkies It is my general understanding that most people would (strongly?) prefer that ddebs were placed in a separate component. I would be surprised if we did ended up including them in the current components. Thanks, ~Niels -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/55545af4.6030...@thykier.net
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On 2015-05-02 13:46, David Kalnischkies wrote: On Fri, May 01, 2015 at 11:46:42PM +0200, Niels Thykier wrote: […] ddeb support […] +1. \o/ - apt now properly handles the pkg:arch dependency. [...] I would revert the revert as this is potentially causing more trouble than the problems it is trying to solve (aka: I don't see why a debug package has to depend on the package it provides symbols for at all. If any the relation should be 'Enhances'…). Best regards David Kalnischkies I add the depends for the following reasons: * It is what we do with manual -dbg packages today and it is what people seem to expect. * It allows me to trivially deploy a doc-symlink to avoid an extra copy of the copyright file to create policy compliant debs. Now, IRT the pkg:arch dependency - it was to ensure that the you get the correct variant of your debug package. I can certainly appreciate that the (original?) Multi-arch spec does not support this for Multi-arch: foreign[1]. We have now ended up in a situation where people has made their own interpretation of how to handle this situation rending pkg:arch dependencies useless when pkg is multi-arch:foreign. It is what happens when people have to guess what something means. :) Thankfully, we got a solution that works perfectly for any other multi-arch value and foreign is just a minor inconvenience when APT guesses wrong on the (architecture for the) debug package. Thanks, ~Niels [1] As I recall, it does not really mention the pkg:arch dependencies. But it is a couple of years since I last read it, so I am quite possibly wrong here. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/55545ea6.5040...@thykier.net
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Sat, May 02, 2015 at 09:07:56AM -0400, James McCoy wrote: On Sat, May 02, 2015 at 01:46:25PM +0200, David Kalnischkies wrote: (aka: I don't see why a debug package has to depend on the package it provides symbols for at all. If any the relation should be 'Enhances'…). The intention is to ensure the debug symbols came from the same build as the binary packages which are being enhanced. Enhances doesn't provide that guarantee since it's a purely aesthetic relationship. The Why? Is something broken by the fact that you have a -dbg(sym) package installed, but not the package the debug symbols are for, or does anything break by having an outdated -dbg(sym) package – appart from your ability to make use of the symbols files? -data packages do not depend on their users even through they are useless without them. -doc packages do not depend on the things they document even through having an outdated version means stuff which is documented to work doesn't (which could be quite dangerous). Note also that the relation you are trying to express is stronger than can be expressed currently as Provides can satisfy such a relation (curtesy of versioned provides being supported by dpkg now), while they are hardly satisfying the intention. I think this Depends is only needed because most manual debug packages have it, so by induction, the automatic ones need it, too. Oh, and I would prefer teaching our packages managers to deal with enhances better, rather than declaring them purely aesthetic forever… There is nothing wrong with teaching them to keep installed (reverse) enhance relations satisfied, much like they do for recommends (and should for suggests) for example. Keeping debug packages autoinstalled, but tagged as non-garbage as long as the package they enhance is installed would be fun, too. There are countless more plans for great things. Unfortuently manpower is too limited to have them all done at once. I hope we can get at least some for stretch… Best regards David Kalnischkies signature.asc Description: Digital signature
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
Hi! On Tue, 2015-04-07 at 22:11:18 +0200, Niels Thykier wrote: A) Use .deb (i.e. the regular extension) with a new section. Is there any problem with using the existing debug section? Or is the different section used to distinguish that these are autogenerated perhaps? B) Use .ddeb (i.e. with an extra d). What where the motivations for using a different extension? I can see the motivation for .udeb:s as they need to live in a different namespace, but that does not seem to be the case for debug debs. Assuming that any issue with debug .deb:s is fixed in the tools, is there any remaining reason to use .ddeb:s? On 2015-04-07 21:10, Niels Thykier wrote: Both have their own advantages and disadvantages. In particular: A) means that almost every existing tool will handle the debug debs like a regular deb (which it is) and will generally work perfectly out of the box. - There are a couple of exceptions, but we are limited to something like lintian and dpkg-genchanges. I'd happily look into making dpkg-genchanges allow this. - There will be tools that might want to handle them differently. Programs like dak and reprepro might want to (support) put(ting) them in a different part of their repositories. They could already do that by keying on the Section, no? Otherwise the filename is also unique enough to be keyed on («*-dbgsym_*»)? - This is *currently not working* since dpkg-genchanges errors out on the auto-generated .deb files. See above. :) B) means that .ddebs can be special cased on filenames rather than on section (like udebs). Furthermore, there might be a lot of things that do not need to support .ddebs at all. Personally I think it breaks expectations, from muscle memory: # dpkg -i *.deb to many tools that might process debs and expect them to have that extension, which of course could be fixed but we have to take into account tools outside of the Debian archive. - Downside is that adding support is a manual extra step for many tools, that (besides the filename) would otherwise be able to handle .ddebs immediately. For example dpkg-scanpackages can be told to use a different package type, but it cannot be told to recognize multiple package types for the same invocation. I guess in this case you can cat the result, but still: $ ( dpkg-scanpackages . ; dpkg-scanpackages --type ddeb . ) Pacakges this would imply either fixing the tool or going around and changing lots of scripts for custom repositories. - On the plus side: dpkg-genchanges in Jessie can support this solution immediately with a minor warning. Sure, immediate deployment is a plus, but I'd rather we'd carefully consider the consequences of any such decission that might be pretty hard to reverse course in the future in case we find it to be too problematic. From my point of view, I am not strongly attached to one solution over the other: * I am slightly preferring A), but I am ready to implement either solution in the tools, I maintain. * The difference for debhelper is a single d and a section name. * The change for lintian is larger, but B) is the heavy solution and I already got a mostly working patch for that. Barring any strong reasoning behind B) above, I pretty much prefer A). Thanks, Guillem -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20150503165831.gb9...@gaara.hadrons.org
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Fri, May 01, 2015 at 11:46:42PM +0200, Niels Thykier wrote: […] ddeb support […] +1. \o/ - apt now properly handles the pkg:arch dependency. For different values of properly – apt isn't the only thing involved here, you have to consider the reaction of dpkg and dose as well and these 3 tools aren't agreeing on the interpretation of pkg:arch in combination with M-A:foreign and various other things at the moment: https://lists.alioth.debian.org/pipermail/multiarch-devel/2015-April/99.html (The initial problem, that you can install packages on single arch system where arch == native-arch which is what debhelper is trying to pull here was fixed, yes, but that doesn't mean that you will actually get a package from the native-arch. -- If anyone wants to comment this, don't do it here, do it in the thread I linked AFTER reading the thread. There is enough complexity and text to fill an afternoon; no need to send a comment in the lunch break with your smartphone…). I would revert the revert as this is potentially causing more trouble than the problems it is trying to solve (aka: I don't see why a debug package has to depend on the package it provides symbols for at all. If any the relation should be 'Enhances'…). Best regards David Kalnischkies signature.asc Description: Digital signature
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Sat, May 02, 2015 at 01:46:25PM +0200, David Kalnischkies wrote:
(aka: I don't see why a debug
package has to depend on the package it provides symbols for at all. If
any the relation should be 'Enhances'…).
The intention is to ensure the debug symbols came from the same build as
the binary packages which are being enhanced. Enhances doesn't provide
that guarantee since it's a purely aesthetic relationship. The
alternative is to have all the packages which are enhanced by the debug
symbols declare “Breaks: foo-dbg ( ${binary:Version}), foo-dbg (
${binary:Version})”
Cheers,
--
James
GPG Key: 4096R/331BA3DB 2011-12-05 James McCoy james...@debian.org
signature.asc
Description: Digital signature
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On 2015-04-04 10:54, Niels Thykier wrote: On 2015-04-04 09:54, Josselin Mouette wrote: Le jeudi 02 avril 2015 à 19:37 +0200, Esokrates a écrit : Hi, I am particularly interested in automatic debug packages, as the current situation is pretty messy imho. I found https://wiki.debian.org/AutomaticDebugPackages. Does anyone know the status of this? Will this be a goal for Stretch? This and reproducible builds would make Debian the perfect distribution for me. Whats in the way of making it happen? Lack of people willing to do it? Last time I checked, dak was still missing code to handle the generated .ddeb files. Cheers, And it *still* does! But there are a few things that have changed! * There is an experimental branch for debhelper to generate these automatically available. [...] Enjoy, ~Niels [...] This branch (with some bug fixes) are now in experimental if anyone wants to play with it. * Remember to set DH_BUILD_DDEBS! Else ddebs will not be built * dak support is still missing * Thanks to the reproducibility team, the APT maintainers and the dpkg maintainers. They have either helped me solve a few bugs or fixed a bug in their respective tools. In particular: - the reproducibility team reported a number of brown paper bag bugs (like dh_strip throwing away debug symbols in regular -dbg pkgs /o\ ). - dpkg-genchanges no longer emits warnings when building .ddebs. - apt now properly handles the pkg:arch dependency. * Some caveats remain (like lack of documentation and there is no valid migration path from -dbg to .ddebs). Thanks, ~Niels -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/5543f442.2010...@thykier.net
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Mon, Apr 20, 2015 at 09:50:00AM +0800, Paul Wise wrote: On Mon, Apr 20, 2015 at 1:10 AM, David Kalnischkies wrote: I would presume most derivatives aren't using it either Most derivatives appear to use reprepro but there is one using apt-ftparchive https://wiki.debian.org/Derivatives/CensusFull I knew it. I had a look at the census page especially as the list had a discussion about it recently, but couldn't find the field there. Not all derivatives seem to declare it and I looked at the wrong ones… I know that a few are using it which haven't declared it. Ubuntu e.g. does (at least that is what I figure from the bugreports). The point was mainly: apt-ftparchive/jessie doesn't support it out of the box, but its easy to change that if someone would need it. (I have my doubts that all others do support it out of the box either) https://wiki.debian.org/Derivatives/Census/Lihuen That is an interesting. The script they use does use the manual steps 'packages' and so on, it does mention 'generate' in a comment through. Gonna talk to them (and d-derivatives@ at large) later. They aren't supporting InRelease nor .xz for example which would be nice if they do. I think there will be some work upon us to make ddebs supported well (I invision something like a apt-get debugsymbols foo which installs the package foo-dbgsym and maybe optionally also the debug packages of the direct dependencies libfoo1 (libfoo-dbgsym) and libbar0.1 (python3-bar-dbgsym as it is the c-binding of a python library as you might (not) have guessed).) but lets get them first, shall we? :) As a user of debug packages I'd like installing foo-dbg to pull in all the -dbg packages I would need to dump a backtrace from GDB. So basically all recursive reverse dependencies. That can be very quickly quite a set of packages. apt ~23, apititude ~40, mpv (similar to mplayer) ~159, kate (KDEs notepad) ~465. [0] That can be tuned by excluding non-libraries, but that has its own drawbacks (private libraries shared between a very closely related set of packages for example), aka: For a quickshot direct dependencies are probably enough (personal observation; the times I needed debug symbols for non-direct dependencies are far and in between, but maybe I am just lucky). If you wanna go fullcircle, its probably better to analyse a core-dump for which symbols are needed exactly instead of getting everything. I think Ubuntu has a tool dubbed apport-retrace (Debian has it in experimental only) which is supposed to do that (but I just remember hearing the name in this context, nothing more) [0] very hacky approximation with apt-cache depends apt --important --no-conflicts --no-replaces --no-breaks --recurse -o APT::Architectures=amd64 -o APT::Cache::ShowOnlyFirstOr=1 | grep -v '^ ' | wc -l Best regards David Kalnischkies signature.asc Description: Digital signature
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Mon, Apr 20, 2015 at 1:10 AM, David Kalnischkies wrote: I would presume most derivatives aren't using it either Most derivatives appear to use reprepro but there is one using apt-ftparchive https://wiki.debian.org/Derivatives/CensusFull https://wiki.debian.org/Derivatives/Census/Lihuen I think there will be some work upon us to make ddebs supported well (I invision something like a apt-get debugsymbols foo which installs the package foo-dbgsym and maybe optionally also the debug packages of the direct dependencies libfoo1 (libfoo-dbgsym) and libbar0.1 (python3-bar-dbgsym as it is the c-binding of a python library as you might (not) have guessed).) but lets get them first, shall we? :) As a user of debug packages I'd like installing foo-dbg to pull in all the -dbg packages I would need to dump a backtrace from GDB. So basically all recursive reverse dependencies. btw: Is it planed to drop them into their own repository/component or are we gone blow up our regular Packages files with them? (you might guess from the wording which way I would prefer). IIRC it was always planned to put them in a separate place. -- bye, pabs https://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CAKTje6GFzSJCyaFwXz06gyUtQc0=5__=1_371_yrdwvm3qg...@mail.gmail.com
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Sat, Apr 04, 2015 at 10:54:09AM +0200, Niels Thykier wrote: The resulting debs are installable with dpkg -i ( \o/ ). I have not tried anything fancy like setting up a local APT mirror and tried to convince APT do install it. I did and apt works with ddeb just fine, meaning it can happily print infos about them, download them and install them with dpkg. There are two exceptions as far as I can see: - apt-ftparchive doesn't know about '.ddeb', so by default neither Contents nor Packages files created by it contain information about them. Users of apt-ftparchive contents/packages are (surprisingly) out of luck as far as I can see and have to wait for a patch (= 2 lines), but users of apt-ftparchive generate can configure the used extension, see apt-ftparchive manpage on how to do that. Debian uses dak to create the archive, so that isn't a problem per-se. I would presume most derivatives aren't using it either as alternatives are plenty. Those who do are very likely using generate as its just strictly better than manually creating Packages files with 'apt-ftparchive packages'. I guess most repository creators have this or similar problems and solutions through and that doesn't seem like all to important for the adoption. - apt/experimental supports installing local .deb files. That doesn't support the '.ddeb' extension yet. I leave it up to the reader to figure out how much of a dealbreaker that is… So, super-cow approves (d)debs. (In fact, apt-dbg never became a thing as automatic ddebs were always very soon now in existence every time it came up. This time please…) And it especially approves the debhelper branch name. ;) I think there will be some work upon us to make ddebs supported well (I invision something like a apt-get debugsymbols foo which installs the package foo-dbgsym and maybe optionally also the debug packages of the direct dependencies libfoo1 (libfoo-dbgsym) and libbar0.1 (python3-bar-dbgsym as it is the c-binding of a python library as you might (not) have guessed).) but lets get them first, shall we? :) btw: Is it planed to drop them into their own repository/component or are we gone blow up our regular Packages files with them? (you might guess from the wording which way I would prefer). Best regards David Kalnischkies signature.asc Description: Digital signature
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On 2015-04-09 09:25, Esokrates wrote: On Tuesday, April 07, 2015 10:11:18 PM Niels Thykier wrote: [...] So mostly that is more a decision making (political) problem, than a technical one. It is not entirely clear to me that we any have (major) political issues IRT ddebs. People generally seem to be positive (or worst case indifferent) to the idea itself. There also seem to be agreement about what goes into the ddebs, where it is placed and they are just another deb (etc.). As mentioned, the only real concern seem to be whether or not it should be a .ddeb or .deb. Though I doubt it is going to be problematic - if this was truly something people felt for strongly, I am certain they would have expressed their opinion by now. Stretch is a two year time frame though, which makes me kinda sad. Thanks for you effort though, keep up the amazing work! If I understand correctly, if it would have been something for stretch, either A or B would have been decided already and partly implemented, right? I believe we can implement this for Stretch if we wanted to. Unlike multi-arch, build-profiles (etc.), ddebs are already supported by our (14-days-from-now-new) stable release. Namely, * All the groundwork requirements (pkg:arch dependencies etc.) are already covered by previous efforts. - I.e. everything that would require a full-cycle for support are already done in time for Jessi * What we are missing is mostly support from our infrastructure and repository tools. As far as I am informed, the main blockers here are: - With .ddebs = dak, debhelper - With .debs = dak, debhelper, dpkg-dev None of which * Known list of tools where support is (very) nice-to-have, but not a strict blocker. - lintian - though people on d-mentors might disagree. ;) - reprepro and similar - others? * List of (assumed) support is completely optional or even unnecessary - buildd/sbuild/wanna-build - it should just be a regular build for these with debhelper doing the heavy lifting. - britney - the ddebs build by debhelper are just as (co-) installable as the .deb they are based on. I am looking forward to the day, when both reproducible builds and automatic debug package exist, that will be an awesome future! Thanks very much for outlining this, Niels! I certainly agree! I cannot take credit for the reproducible builds though. :) [Different mail, same sender]: Or maybe another question: Will it be ready for sid, soon? Sorry, I am really excited about this. Good question really. I can certainly upload a version of debhelper to unstable that has /opt-in/ support for ddebs when Stretch opens. However, to have them on the mirror largely depends on when dak gets support for ddebs. Unfortunately, I do not have any ETA on that. Thanks, ~Niels -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/5528adb6.8050...@thykier.net
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Tuesday, April 07, 2015 10:11:18 PM Niels Thykier wrote: On 2015-04-07 21:10, Niels Thykier wrote: On 2015-04-04 12:58, Esokrates wrote: On Saturday, April 04, 2015 10:54:09 AM Niels Thykier wrote: [...] I know predictions are hard, but is there a plan to get things done for the next release (Stretch)? At this point, there is no plan, sorry. However, we got a functional prototype (for part of the problem) and some people poking a bit at it from a design view. I received conflicting remarks: A) Use .ddeb (i.e. with an extra d). B) Use .deb (i.e. the regular extension) with a new section. I managed to confuse myself here and swapped A and B in the above. What I meant to write was: A) Use .deb (i.e. the regular extension) with a new section. B) Use .ddeb (i.e. with an extra d). The rest should now make sense - apologies for the confusion: Both have their own advantages and disadvantages. In particular: A) means that almost every existing tool will handle the debug debs like a regular deb (which it is) and will generally work perfectly out of the box. - There are a couple of exceptions, but we are limited to something like lintian and dpkg-genchanges. - There will be tools that might want to handle them differently. Programs like dak and reprepro might want to (support) put(ting) them in a different part of their repositories. - This is *currently not working* since dpkg-genchanges errors out on the auto-generated .deb files. B) means that .ddebs can be special cased on filenames rather than on section (like udebs). Furthermore, there might be a lot of things that do not need to support .ddebs at all. - Downside is that adding support is a manual extra step for many tools, that (besides the filename) would otherwise be able to handle .ddebs immediately. - On the plus side: dpkg-genchanges in Jessie can support this solution immediately with a minor warning. From my point of view, I am not strongly attached to one solution over the other: * I am slightly preferring A), but I am ready to implement either solution in the tools, I maintain. * The difference for debhelper is a single d and a section name. * The change for lintian is larger, but B) is the heavy solution and I already got a mostly working patch for that. [...] So mostly that is more a decision making (political) problem, than a technical one. Stretch is a two year time frame though, which makes me kinda sad. Thanks for you effort though, keep up the amazing work! If I understand correctly, if it would have been something for stretch, either A or B would have been decided already and partly implemented, right? I am looking forward to the day, when both reproducible builds and automatic debug package exist, that will be an awesome future! Thanks very much for outlining this, Niels! -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/3487072.50TJyGfKk4@ubuntu
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Thursday, April 09, 2015 09:25:44 AM Esokrates wrote: So mostly that is more a decision making (political) problem, than a technical one. Stretch is a two year time frame though, which makes me kinda sad. Thanks for you effort though, keep up the amazing work! If I understand correctly, if it would have been something for stretch, either A or B would have been decided already and partly implemented, right? I am looking forward to the day, when both reproducible builds and automatic debug package exist, that will be an awesome future! Thanks very much for outlining this, Niels! Or maybe another question: Will it be ready for sid, soon? Sorry, I am really excited about this. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/3804029.DFJoepXrID@ubuntu
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On 2015-04-04 12:58, Esokrates wrote: On Saturday, April 04, 2015 10:54:09 AM Niels Thykier wrote: [...] - Trying to get the reproducible team to try it out to see if it regresses anything (incl. reproducible builds) I guess the ddeb's are meant to be reproducible too? Yes. The .ddebs are currently being built by the reproducibility team on jenkins.debian.net and they are indeed reproducible themselves (after we extended a patch the reproducible debhelper patch series to account for them IRT mtimes). - It *does* cause dpkg-genchanges to emit warnings about uninitialized values (I think 3 per ddeb). Related to #781074 - DOES *NOT* PRODUCE DDEBS FOR PACKAGES COVERED BY AN EXISTING -dbg! Sounds like you plan to keep -dbg packages? Or just for migration? Ubuntu kept -dbg packages and tries to maintain -dbgsym along it (but they only cover packages that no -dbg package exists for, otherwise it will be an empty package, no idea if they situation is still like that there ...), however the situation was kind of messy, I often got file conflicts. Things were not consistent at all last time I needed debug packages for a lot of stuff there. Simply put: If you installed the debug package for every installed package, you get the hell of a mess, this was almost undoable. (That test maybe does not make a lot of sense, but it did show that things were not optimal.) I have no particular interest in keeping existing manual -dbg packages. However, there seemed to be little point in creating a .ddeb which is a perfect clone of an existing -dbg with neither of them being co-installable. Please also note that there are some existing manual debug packages that we *cannot* migrate to ddeb currently (at least not as-is). E.g. perl-debug provides debug symbols but also a debugperl binary etc. So the prototype patch plays it safe and only generate .ddebs if the debug symbols would otherwise have been discarded. This also means that if the -dbg does not cover all binaries, debhelper will generate .ddebs for the uncovered binaries (where applicable). - DOES *NOT* ADD BREAKS/REPLACES FOR MIGRATING FROM -dbg TO .ddeb! For reference, this limitation is due to time constraints - I intend to have a solution for this problem (but it will probably have to be manual - maybe an argument for dh_gencontrol). [...] I know predictions are hard, but is there a plan to get things done for the next release (Stretch)? At this point, there is no plan, sorry. However, we got a functional prototype (for part of the problem) and some people poking a bit at it from a design view. I received conflicting remarks: A) Use .ddeb (i.e. with an extra d). B) Use .deb (i.e. the regular extension) with a new section. Both have their own advantages and disadvantages. In particular: A) means that almost every existing tool will handle the debug debs like a regular deb (which it is) and will generally work perfectly out of the box. - There are a couple of exceptions, but we are limited to something like lintian and dpkg-genchanges. - There will be tools that might want to handle them differently. Programs like dak and reprepro might want to (support) put(ting) them in a different part of their repositories. - This is *currently not working* since dpkg-genchanges errors out on the auto-generated .deb files. B) means that .ddebs can be special cased on filenames rather than on section (like udebs). Furthermore, there might be a lot of things that do not need to support .ddebs at all. - Downside is that adding support is a manual extra step for many tools, that (besides the filename) would otherwise be able to handle .ddebs immediately. - On the plus side: dpkg-genchanges in Jessie can support this solution immediately with a minor warning. From my point of view, I am not strongly attached to one solution over the other: * I am slightly preferring A), but I am ready to implement either solution in the tools, I maintain. * The difference for debhelper is a single d and a section name. * The change for lintian is larger, but B) is the heavy solution and I already got a mostly working patch for that. Thanks for the very comprehensive status update and your awesome work! You are welcome. :) Thanks, ~Niels -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/55242b97.5080...@thykier.net
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On 2015-04-07 21:10, Niels Thykier wrote: On 2015-04-04 12:58, Esokrates wrote: On Saturday, April 04, 2015 10:54:09 AM Niels Thykier wrote: [...] I know predictions are hard, but is there a plan to get things done for the next release (Stretch)? At this point, there is no plan, sorry. However, we got a functional prototype (for part of the problem) and some people poking a bit at it from a design view. I received conflicting remarks: A) Use .ddeb (i.e. with an extra d). B) Use .deb (i.e. the regular extension) with a new section. I managed to confuse myself here and swapped A and B in the above. What I meant to write was: A) Use .deb (i.e. the regular extension) with a new section. B) Use .ddeb (i.e. with an extra d). The rest should now make sense - apologies for the confusion: Both have their own advantages and disadvantages. In particular: A) means that almost every existing tool will handle the debug debs like a regular deb (which it is) and will generally work perfectly out of the box. - There are a couple of exceptions, but we are limited to something like lintian and dpkg-genchanges. - There will be tools that might want to handle them differently. Programs like dak and reprepro might want to (support) put(ting) them in a different part of their repositories. - This is *currently not working* since dpkg-genchanges errors out on the auto-generated .deb files. B) means that .ddebs can be special cased on filenames rather than on section (like udebs). Furthermore, there might be a lot of things that do not need to support .ddebs at all. - Downside is that adding support is a manual extra step for many tools, that (besides the filename) would otherwise be able to handle .ddebs immediately. - On the plus side: dpkg-genchanges in Jessie can support this solution immediately with a minor warning. From my point of view, I am not strongly attached to one solution over the other: * I am slightly preferring A), but I am ready to implement either solution in the tools, I maintain. * The difference for debhelper is a single d and a section name. * The change for lintian is larger, but B) is the heavy solution and I already got a mostly working patch for that. [...] -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/552439e6.40...@thykier.net
Re: -dbg packages; are they actually useful?
Le jeudi 02 avril 2015 à 19:37 +0200, Esokrates a écrit : Hi, I am particularly interested in automatic debug packages, as the current situation is pretty messy imho. I found https://wiki.debian.org/AutomaticDebugPackages. Does anyone know the status of this? Will this be a goal for Stretch? This and reproducible builds would make Debian the perfect distribution for me. Whats in the way of making it happen? Lack of people willing to do it? Last time I checked, dak was still missing code to handle the generated .ddeb files. Cheers, -- .''`. Josselin Mouette : :' : `. `' `- -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/1428134082.8479.3.ca...@debian.org
Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On 2015-04-04 09:54, Josselin Mouette wrote: Le jeudi 02 avril 2015 à 19:37 +0200, Esokrates a écrit : Hi, I am particularly interested in automatic debug packages, as the current situation is pretty messy imho. I found https://wiki.debian.org/AutomaticDebugPackages. Does anyone know the status of this? Will this be a goal for Stretch? This and reproducible builds would make Debian the perfect distribution for me. Whats in the way of making it happen? Lack of people willing to do it? Last time I checked, dak was still missing code to handle the generated .ddeb files. Cheers, And it *still* does! But there are a few things that have changed! * There is an experimental branch for debhelper to generate these automatically available. - Requires a export DH_BUILD_DDEBS=1 to trigger the code path - It applies to *all* compat levels. - Trying to get the reproducible team to try it out to see if it regresses anything (incl. reproducible builds) - It *does* cause dpkg-genchanges to emit warnings about uninitialized values (I think 3 per ddeb). Related to #781074 - DOES *NOT* PRODUCE DDEBS FOR PACKAGES COVERED BY AN EXISTING -dbg! - DOES *NOT* ADD BREAKS/REPLACES FOR MIGRATING FROM -dbg TO .ddeb! - Branch at [DH-BRANCH] * Half-done branch for lintian to actually look at ddebs - Known false-positive triggered for ddebs (output says FIXME, so it should be trivial to spot). - Branch at [LINT-BRANCH] The resulting debs are installable with dpkg -i ( \o/ ). I have not tried anything fancy like setting up a local APT mirror and tried to convince APT do install it. *Known to be missing*: * Documentation in debhelper (incl. changelog entries) * A solution to the -dbg = .ddeb migration, so you can do a -2 upload that uses .ddeb instead of -dbg and *not* get file conflicts. * FTP master approval + dak support - Possibly a solution that does not double the size of Packages. * APT support or verification that it works out of the box - help welcome. * dpkg (dpkg-genchanges) should pref. not emit uninitialised warnings - Probably related #781074. My guess is that a call to debarch_eq is the source of all warnings. I have not pinpointed which call. ... and I am late for something now, so I will cut it short here. Enjoy, ~Niels [DH-BRANCH]: https://anonscm.debian.org/cgit/users/nthykier/debhelper.git/log/?h=super-cow-ddebs-support [LINT-BRANCH]: http://anonscm.debian.org/cgit/users/nthykier/lintian.git/log/?h=ddeb-support -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/551fa6b1.9080...@thykier.net
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Saturday, April 04, 2015 10:54:09 AM Niels Thykier wrote: Last time I checked, dak was still missing code to handle the generated .ddeb files. Cheers, And it *still* does! But there are a few things that have changed! * There is an experimental branch for debhelper to generate these automatically available. - Requires a export DH_BUILD_DDEBS=1 to trigger the code path - It applies to *all* compat levels. - Trying to get the reproducible team to try it out to see if it regresses anything (incl. reproducible builds) I guess the ddeb's are meant to be reproducible too? - It *does* cause dpkg-genchanges to emit warnings about uninitialized values (I think 3 per ddeb). Related to #781074 - DOES *NOT* PRODUCE DDEBS FOR PACKAGES COVERED BY AN EXISTING -dbg! Sounds like you plan to keep -dbg packages? Or just for migration? Ubuntu kept -dbg packages and tries to maintain -dbgsym along it (but they only cover packages that no -dbg package exists for, otherwise it will be an empty package, no idea if they situation is still like that there ...), however the situation was kind of messy, I often got file conflicts. Things were not consistent at all last time I needed debug packages for a lot of stuff there. Simply put: If you installed the debug package for every installed package, you get the hell of a mess, this was almost undoable. (That test maybe does not make a lot of sense, but it did show that things were not optimal.) - DOES *NOT* ADD BREAKS/REPLACES FOR MIGRATING FROM -dbg TO .ddeb! - Branch at [DH-BRANCH] * Half-done branch for lintian to actually look at ddebs - Known false-positive triggered for ddebs (output says FIXME, so it should be trivial to spot). - Branch at [LINT-BRANCH] The resulting debs are installable with dpkg -i ( \o/ ). I have not tried anything fancy like setting up a local APT mirror and tried to convince APT do install it. *Known to be missing*: * Documentation in debhelper (incl. changelog entries) * A solution to the -dbg = .ddeb migration, so you can do a -2 upload that uses .ddeb instead of -dbg and *not* get file conflicts. * FTP master approval + dak support - Possibly a solution that does not double the size of Packages. * APT support or verification that it works out of the box - help welcome. * dpkg (dpkg-genchanges) should pref. not emit uninitialised warnings - Probably related #781074. My guess is that a call to debarch_eq is the source of all warnings. I have not pinpointed which call. ... and I am late for something now, so I will cut it short here. Enjoy, ~Niels [DH-BRANCH]: https://anonscm.debian.org/cgit/users/nthykier/debhelper.git/log/?h=super-co w-ddebs-support [LINT-BRANCH]: http://anonscm.debian.org/cgit/users/nthykier/lintian.git/log/?h=ddeb-suppor t I know predictions are hard, but is there a plan to get things done for the next release (Stretch)? Thanks for the very comprehensive status update and your awesome work! -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/2478425.nmYp5U8jSL@ubuntu
Re: -dbg packages; are they actually useful?
Hi, I am particularly interested in automatic debug packages, as the current situation is pretty messy imho. I found https://wiki.debian.org/AutomaticDebugPackages. Does anyone know the status of this? Will this be a goal for Stretch? This and reproducible builds would make Debian the perfect distribution for me. Whats in the way of making it happen? Lack of people willing to do it? I hope you can enlighten me, Thanks everyone for their awesome work! -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/3122558.HgHPhVL5Wy@ubuntu
Re: -dbg packages; are they actually useful?
Hi, Even Microsoft has a service for downloading symbols files for many core windows components on demand (integrated with the crash dump analysis tool. (I forget of the chrash dump tool is part of the pre-installed debugger or it itis seperate)). They even go a step further and ask for application vendors to submit the .pdb files for their software, and forward crash reports. The biggest problem I see with such a service would be matching the debug information to the binaries on the user's system. MS use a GUID for each loadable object AFAIK, which could probably be recreated easily -- still leaves the problem that in order for the service to be useful, we need quite a back archive of debug information for older versions. Simon signature.asc Description: Digital signature
Re: -dbg packages; are they actually useful?
On Wed, Mar 04, 2009 at 10:19:22PM -0800, Steve Langasek wrote: On Wed, Mar 04, 2009 at 10:03:28AM +, Tzafrir Cohen wrote: On Tue, Mar 03, 2009 at 09:17:17PM -0800, Steve Langasek wrote: Remaining concerns: - each of these dbg packages requires manual modification to the source package (incl. adding the package to debian/control) - each has to go through the NEW queue - each takes up space afterwards in the Packages file Much better if these can be generated centrally as part of the builds. What about backports? What about locally-built packages? What about them? Are you suggesting that we should instead continue to manually add -dbg packages to all source packages for the benefit of people who aren't even using our binary packages? Maybe. I'm only stating the problems I encounter. (Sorry, we can't help you debug your probelem until you ditch that package you built and build from source like Real Men should). Or they could: - build their packages with DEB_BUILD_OPTIONS=nostrip, the standard mechanism for building packages containing unstripped binaries, or - use the pkgbinarymangler script, available as a package from Ubuntu, to autogenerate debug packages from any debhelper-using package as part of the build It means I have to rebuild the package. And hope I have a similar enough build environment to the one originally used. With rpm there's no such problem, as a separate debug package is created automatically. I just have to keep it somewhere. -- Tzafrir Cohen | tzaf...@jabber.org | VIM is http://tzafrir.org.il || a Mutt's tzaf...@cohens.org.il || best ICQ# 16849754 || friend -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
On Thu, Mar 5, 2009 at 5:23 PM, Tzafrir Cohen tzaf...@cohens.org.il wrote: With rpm there's no such problem, as a separate debug package is created automatically. I just have to keep it somewhere. That is exactly the plan for debug.debian.net. IMO it needs to sidestep dh_strip though, since debhelper isn't mandatory and isn't used by all packages. -- bye, pabs http://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
On Tue, Mar 03, 2009 at 10:25:06PM -0500, Theodore Tso wrote: On Tue, Mar 03, 2009 at 10:12:22PM +, Steve McIntyre wrote: I'm looking at my local mirror (slowly) update at the moment, and I've got to wondering: are the large -dbg packages actually really useful to anybody? I can't imagine that more than a handful of users ever install (to pick an example) the amarok-dbg packages, but we have multiple copies of a 70MB-plus .deb taking up mirror space and bandwidth. I can understand this for library packages, maybe, but for applications? There are people working on ways of compressing the debuginfo information, and I've been told they might have results within a couple of months. Part of the problem is that depending on how the package is built, the -dbg packages can be huge, so it makes the cost/benefit ratio somewhat painful. Compressing -dbg files using dh_builddeb -Zlzma, which uses lzma compression instead of gzip, gives an average gain of 1.88 in size for the current -dbg packages we have in sid. -- Aurelien Jarno GPG: 1024D/F1BCDB73 aurel...@aurel32.net http://www.aurel32.net -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
Aurelien Jarno aurel...@aurel32.net writes: Compressing -dbg files using dh_builddeb -Zlzma, which uses lzma compression instead of gzip, gives an average gain of 1.88 in size for the current -dbg packages we have in sid. Compressing with bzip2 is already supported by DAK and might help almost as much. (I realize that LZMA is a better compression scheme overall, but it's apparently also deprecated, replaced with XZ, and using it in the archive will require further DAK changes or work to switch to XZ instead.) -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
On Thu, Mar 05, 2009 at 05:35:53PM +0900, Paul Wise wrote: On Thu, Mar 5, 2009 at 5:23 PM, Tzafrir Cohen tzaf...@cohens.org.il wrote: With rpm there's no such problem, as a separate debug package is created automatically. I just have to keep it somewhere. That is exactly the plan for debug.debian.net. So, back to what I originally asked about: * Locally-generated packages? * Backports? * ( Any other semi- and non-official repository) Dear packagers, if your package is not intended for upload, please be sure not to strip debug symbols. -- Tzafrir Cohen | tzaf...@jabber.org | VIM is http://tzafrir.org.il || a Mutt's tzaf...@cohens.org.il || best ICQ# 16849754 || friend -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
On Thu, Mar 05, 2009 at 12:43:43AM -0800, Russ Allbery wrote: Aurelien Jarno aurel...@aurel32.net writes: Compressing -dbg files using dh_builddeb -Zlzma, which uses lzma compression instead of gzip, gives an average gain of 1.88 in size for the current -dbg packages we have in sid. Compressing with bzip2 is already supported by DAK and might help almost as much. (I realize that LZMA is a better compression scheme overall, but Most of the tests shows that bzip2 is slower to decompress an archive and has a smaller gain than LZMA. it's apparently also deprecated, replaced with XZ, and using it in the archive will require further DAK changes or work to switch to XZ instead.) AFAIK, allowing LZMA in DAK is just a matter of changing one line in DAK. Do you have more details about LZMA being deprecated and replaced by XZ? If we want to allow XZ, it will be for squeeze + 1 only. BTW, I think we should also to disallow the compression for some Debian packages. There is no point of compressing a package with gzip when it contains a single source.bz2 (or .gz or .lzma) and a changelog.gz. This is already supported by the tools, but not by DAK. -- Aurelien Jarno GPG: 1024D/F1BCDB73 aurel...@aurel32.net http://www.aurel32.net -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
Aurelien Jarno aurel...@aurel32.net writes: AFAIK, allowing LZMA in DAK is just a matter of changing one line in DAK. Do you have more details about LZMA being deprecated and replaced by XZ? See current state of the development at http://tukaani.org/lzma/ and then http://tukaani.org/xz/ and note in particular this part of the release announcement of the latest GNU coreutils package: Distribution note: new suffix, .xz replaces .lzma. Note the better-compressed tar ball name below has the .xz suffix. XZ Utils (new name for the improved format/tools) is the successor to lzma. For more info, see http://tukaani.org/xz/ It's possible that XZ archives can be uncompressed with LZMA. I haven't investigated further. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
Quoting Steve Langasek (vor...@debian.org): symbols. If there's a will to get that done in Debian now, I will definitely be happy to ditch the samba-dbg package for one. I support my co-maintainer on that..:-) One should note that samba-dbg is sometimes used and already allowed tracking down some segfault bugs, which are pretty common with samba packages as smbd panics trigger a mail to root that recommends sending a bug report...if the panic is reproducible in some way. But, indeed, learning about the system that's in Ubuntu is great. By coincidence, this is time for GSOC proposals. Wouldn't be something like implement a backtrace anaylysing system similar to Ubuntu's something that could be useful. Alternative: implement ddebs.debian.org signature.asc Description: Digital signature
Re: -dbg packages; are they actually useful?
Steve McIntyre st...@einval.com writes: Hey folks, I'm looking at my local mirror (slowly) update at the moment, and I've got to wondering: are the large -dbg packages actually really useful to anybody? I can't imagine that more than a handful of users ever install (to pick an example) the amarok-dbg packages, but we have multiple copies of a 70MB-plus .deb taking up mirror space and bandwidth. I can understand this for library packages, maybe, but for applications? Thoughts? I think the debug packages are quite usefull. Just not every day or everyone. For a local mirror I would totaly filter out all the -dbg packages. If the need arises you can always fetch them from debian or alter the filter to allow some in. The problem is that when you need the -dbg package then it has to be available. They can not be build after for example you (as maintainer) recieved a core dump. So for Debian they need to be around. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
On 2009-03-04, Steve Langasek vor...@debian.org wrote: [snip] What I really wish for is the ability to have a relatively centralized location where the symbols from every single package ended up that was separate from the normal mirrors. Yes, absolutely. Doing this right, though, requires integration with the buildd network, so that the debugging symbols can be extracted as part of the official build instead of being lossily reconstructed after the fact. I'd like to see input from a ftpmaster here. Let's hope that we see a new host with large storage soon. I've got 10G/suite/arch quoted from Martin Pitt as an upper bound[*] based on his observations on Ubuntu. As we want a data.debian.org anyway, where people could fetch large stuff fast it would make sense IMHO to put those debug packages there too. dak could easily extract it from the changes and send it to the other host for serving, or they could be transmitted separately. I'd be all open to try this on a few arches on the official buildd net. If the experiment proves successful we could drop the -dbg packages to save space on the normal archive mirrors. From my own observation I found the apport retracting in Launchpad pretty helpful as you get meaningful stack traces without user intervention. On the other hand integrating that is an even bigger beast. Kind regards, Philipp Kern [*] The real count currently looks more like 5G/release/arch, but this might not be full coverage due to hickups of the cronjob. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
On Wed, Mar 4, 2009 at 7:12 AM, Christian Perrier bubu...@debian.org wrote: Quoting Steve Langasek (vor...@debian.org): symbols. If there's a will to get that done in Debian now, I will definitely be happy to ditch the samba-dbg package for one. I support my co-maintainer on that..:-) One should note that samba-dbg is sometimes used and already allowed tracking down some segfault bugs, which are pretty common with samba packages as smbd panics trigger a mail to root that recommends sending a bug report...if the panic is reproducible in some way. But, indeed, learning about the system that's in Ubuntu is great. By coincidence, this is time for GSOC proposals. Wouldn't be something like implement a backtrace anaylysing system similar to Ubuntu's something that could be useful. They are already bug 508585 please cc :) And improve it. But I agree it will be nice to have it for next realease. Alternative: implement ddebs.debian.org Yes and for all arch. Sparc is really an interesting arch particularly when you use floatting point. Regards Bastien -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
On Wed, Mar 4, 2009 at 6:17 AM, Steve Langasek vor...@debian.org wrote: On Tue, Mar 03, 2009 at 10:25:06PM -0500, Theodore Tso wrote: I'm looking at my local mirror (slowly) update at the moment, and I've got to wondering: are the large -dbg packages actually really useful to anybody? I can't imagine that more than a handful of users ever install (to pick an example) the amarok-dbg packages, but we have multiple copies of a 70MB-plus .deb taking up mirror space and bandwidth. I can understand this for library packages, maybe, but for applications? There are people working on ways of compressing the debuginfo information, and I've been told they might have results within a couple of months. Part of the problem is that depending on how the package is built, the -dbg packages can be huge, so it makes the cost/benefit ratio somewhat painful. If the -dbg files were more like these sizes: 224 e2fslibs-dbg_1.41.3-1_i386.deb 52 libss2-dbg_1.41.3-1_i386.deb 452 e2fsprogs-dbg_1.41.3-1_i386.deb 48 libuuid1-dbg_1.41.3-1_i386.deb 76 libblkid1-dbg_1.41.3-1_i386.deb 48 uuid-runtime-dbg_1.41.3-1_i386.deb 44 libcomerr2-dbg_1.41.3-1_i386.deb I doubt there's be too much concern Remaining concerns: - each of these dbg packages requires manual modification to the source package (incl. adding the package to debian/control) - each has to go through the NEW queue - each takes up space afterwards in the Packages file Much better if these can be generated centrally as part of the builds. Yes like for instance http://debug.debian.net/ limited unfortunatly to i386 and not up to date, see also #508585. It is really important for us in order to get nice bactrace. I maintain for instance imagemagick and it will be really nice to ask user to do an apt-get debug imagemagick in order to get a reliable backtrace. Moreover this operation should not be priveligied, simple joe user should be able to use debug package. Moreover, apt-get debugdepend imagemagick should also include debug package for dependancies of imagemagick in case of crash in library used by imagemagick. And a script like ddebug could ease user report. For instance the user will do ddebug imagemagick and the script will get a backstrace and generate a bug report. Regards Bastien -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
On Wed, Mar 04, 2009 at 01:45:38PM +0100, Bastien ROUCARIES wrote: Remaining concerns: - each of these dbg packages requires manual modification to the source package (incl. adding the package to debian/control) - each has to go through the NEW queue - each takes up space afterwards in the Packages file Much better if these can be generated centrally as part of the builds. Yes like for instance http://debug.debian.net/ limited unfortunatly to i386 and not up to date, And also inherently unreliable, because these symbols are from different builds than the main binary packages. Have you run into many problems of this sort, where the symbols don't actually match up with the packages in the archive? -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
Don Armstrong wrote: On Tue, 03 Mar 2009, Steve McIntyre wrote: I've got to wondering: are the large -dbg packages actually really useful to anybody? Thoughts? I think they are useful, but probably not for the vast majority of users. [I've used them on a few dozen occasions.] What I really wish for is the ability to have a relatively centralized location where the symbols from every single package ended up that was separate from the normal mirrors. Even Microsoft has a service for downloading symbols files for many core windows components on demand (integrated with the crash dump analysis tool. (I forget of the chrash dump tool is part of the pre-installed debugger or it itis seperate)). It would be really nice to have a similar service. I suppose that if the official buildds would keep the debugging data around using a technique like debug.debian.net uses, and debug.debian.net uses that data, we would be 95% of the way there. Then all we would need is some nice convienece scripts and the deprecation of the -dbg packages. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
On Wed, Mar 04, 2009 at 10:03:28AM +, Tzafrir Cohen wrote: On Tue, Mar 03, 2009 at 09:17:17PM -0800, Steve Langasek wrote: Remaining concerns: - each of these dbg packages requires manual modification to the source package (incl. adding the package to debian/control) - each has to go through the NEW queue - each takes up space afterwards in the Packages file Much better if these can be generated centrally as part of the builds. What about backports? What about locally-built packages? What about them? Are you suggesting that we should instead continue to manually add -dbg packages to all source packages for the benefit of people who aren't even using our binary packages? (Sorry, we can't help you debug your probelem until you ditch that package you built and build from source like Real Men should). Or they could: - build their packages with DEB_BUILD_OPTIONS=nostrip, the standard mechanism for building packages containing unstripped binaries, or - use the pkgbinarymangler script, available as a package from Ubuntu, to autogenerate debug packages from any debhelper-using package as part of the build -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
Steve McIntyre st...@einval.com writes: I'm looking at my local mirror (slowly) update at the moment, and I've got to wondering: are the large -dbg packages actually really useful to anybody? I can't imagine that more than a handful of users ever install (to pick an example) the amarok-dbg packages, but we have multiple copies of a 70MB-plus .deb taking up mirror space and bandwidth. I can understand this for library packages, maybe, but for applications? They've been vital for me several times with library packages and I've occasionally cursed libraries that didn't have them. I find them much less interesting for applications (and indeed dropped them from one application package that I took over after it was orphaned). There are some exceptions, though; for example, I ship debug symbols for the OpenAFS fileserver since it's often hard to figure out what's going on without a backtrace and upstream is very active and very good about analyzing those backtraces. I similarly think it's important to provide debugging symbols for slapd. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
On Tue, 03 Mar 2009, Steve McIntyre wrote: I've got to wondering: are the large -dbg packages actually really useful to anybody? Thoughts? I think they are useful, but probably not for the vast majority of users. [I've used them on a few dozen occasions.] What I really wish for is the ability to have a relatively centralized location where the symbols from every single package ended up that was separate from the normal mirrors. The above, coupled with a coredump submission site which would accept coredumps and automatically generate backtraces for them (or a script that downloaded the -dbg packages, unpacked them and backtraced the coredump) would be a great help in debugging some of the relatively rare segfaults. [We could probably even hook up a coredump handler to such a script.] There was some talk that Ubuntu was going to implement such a thing at the Prague UDS, but I've no clue if it ever came to fruition. Don Armstrong -- Science is a way of trying not to fool yourself. The first principle is that you must not fool yourself, and you are the easiest person to fool. -- Richard Feynman What is and What Should be the Role of Scientific Culture in Modern Society; 1964 http://www.donarmstrong.com http://rzlab.ucr.edu -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
On Wed, Mar 4, 2009 at 12:11 AM, Don Armstrong d...@debian.org wrote: On Tue, 03 Mar 2009, Steve McIntyre wrote: I've got to wondering: are the large -dbg packages actually really useful to anybody? Thoughts? See #508585 and http://debug.debian.net/ It will be really nice to have this stuff generalized for squeeze. I think they are useful, but probably not for the vast majority of users. [I've used them on a few dozen occasions.] What I really wish for is the ability to have a relatively centralized location where the symbols from every single package ended up that was separate from the normal mirrors. See http://debug.debian.net/ The above, coupled with a coredump submission site which would accept coredumps and automatically generate backtraces for them (or a script that downloaded the -dbg packages, unpacked them and backtraced the coredump) would be a great help in debugging some of the relatively rare segfaults. [We could probably even hook up a coredump handler to such a script.] Like unbuntu system. it is really really helpful. There was some talk that Ubuntu was going to implement such a thing at the Prague UDS, but I've no clue if it ever came to fruition. It is implemented in ubuntu, and lauchpad receive automatic report. Don Armstron -- Bastien -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
Hi, Don Armstrong d...@debian.org writes: What I really wish for is the ability to have a relatively centralized location where the symbols from every single package ended up that was separate from the normal mirrors. The above, coupled with a coredump submission site which would accept coredumps and automatically generate backtraces for them (or a script that downloaded the -dbg packages, unpacked them and backtraced the coredump) would be a great help in debugging some of the relatively rare segfaults. [We could probably even hook up a coredump handler to such a script.] There was some talk that Ubuntu was going to implement such a thing at the Prague UDS, but I've no clue if it ever came to fruition. Ubuntu has both of the above: automatic generation of debug symbol packages at build time [1] and a backtracing service integrated in the bug tracker [2]. Regards, Ansgar [1] https://lists.ubuntu.com/archives/ubuntu-devel-announce/2006-September/000195.html Packages with debug symbols can be downloaded from http://ddebs.ubuntu.com [2] https://lists.ubuntu.com/archives/ubuntu-devel/2007-March/023440.html -- PGP: 1024D/595FAD19 739E 2D09 0969 BEA9 9797 B055 DDB0 2FF7 595F AD19 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
On 2009-03-03, Steve McIntyre st...@einval.com wrote: Hey folks, I'm looking at my local mirror (slowly) update at the moment, and I've got to wondering: are the large -dbg packages actually really useful to anybody? I can't imagine that more than a handful of users ever install (to pick an example) the amarok-dbg packages, but we have multiple copies of a 70MB-plus .deb taking up mirror space and bandwidth. I can understand this for library packages, maybe, but for applications? Yes. They are very useful - without those, crash reports are mostly useless. /Sune - thru his KDE maintainer glasses -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
Hi, On Tue, Mar 03, 2009 at 10:12:22PM +, Steve McIntyre wrote: I'm looking at my local mirror (slowly) update at the moment, and I've got to wondering: are the large -dbg packages actually really useful to anybody? I can't imagine that more than a handful of users ever install (to pick an example) the amarok-dbg packages, but we have multiple copies of a 70MB-plus .deb taking up mirror space and bandwidth. I can understand this for library packages, maybe, but for applications? I was glad to have them available for e.g. tracking down some nasty xulrunner bugs on non-x86 -- and I guess xulrunner only counts half as library. Same for liferea, webkit (a library, okay: :) and some webkit browsers (midori comes to my mind). Just an idea coming to my mind: What if they are available from their own APT repository which doesn't need to be mirrored everywhere? Somehow in a similar way as the CD images are distributed separately. Ok, the CD images are no APT repository and splitting off the -dbg packages to a separate repository would mean that what is built from one source package would have to be split up (probably after upload) and put into different APT repositories. Regards, Axel -- Axel Beckert - a...@deuxchevaux.org, a...@noone.org - http://noone.org/abe/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
I doubt most users will install them on their own, but I've found them to be moderately useful in tracking down crashes. It's easier to convince people to install a -dbg package than to convince them to recompile the program from source. Daniel -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
On Tue, Mar 03, 2009 at 03:11:12PM -0800, Don Armstrong wrote: On Tue, 03 Mar 2009, Steve McIntyre wrote: I've got to wondering: are the large -dbg packages actually really useful to anybody? Thoughts? I think they are useful, but probably not for the vast majority of users. [I've used them on a few dozen occasions.] There are 785 packages matching '*-dbg' in unstable on i386. 327 of them are for applications (well, !libraries). Does it really make sense to ship all of these in the archive if, out of the whole set, they're useful to people on a few dozen occasions? According to popcon[1], 433 of these packages have an install count of 10 or less; 616 have an install count of 30 or less.[2] For orphaned packages, these are the kinds of numbers where the QA team starts talking about removals. Granted, the numbers on -dbg packages are going to be lower because they're often installed just for debugging and then removed again, but I think we should seriously look at whether all these one-off debug builds are really justified, and whether they really belong as part of the main archive (and on all our mirrors). What I really wish for is the ability to have a relatively centralized location where the symbols from every single package ended up that was separate from the normal mirrors. Yes, absolutely. Doing this right, though, requires integration with the buildd network, so that the debugging symbols can be extracted as part of the official build instead of being lossily reconstructed after the fact. The above, coupled with a coredump submission site which would accept coredumps and automatically generate backtraces for them (or a script that downloaded the -dbg packages, unpacked them and backtraced the coredump) would be a great help in debugging some of the relatively rare segfaults. [We could probably even hook up a coredump handler to such a script.] There was some talk that Ubuntu was going to implement such a thing at the Prague UDS, but I've no clue if it ever came to fruition. 'apport' in Ubuntu does exactly this (and has been in use since well before the Prague UDS); it hasn't really been worth evaluating for inclusion in Debian without first resolving the problem of lack of systematic debugging symbols. If there's a will to get that done in Debian now, I will definitely be happy to ditch the samba-dbg package for one. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org [1] Well, popcon also thinks there are 887 such packages, rather than 785; I guess there are some of these no longer in unstable. [2] Unfortunately, thanks to bug-buddy Recommending gnome-dbg, we also have almost 40 GNOME -dbg packages with greater popcon stats than libc6-dbg! -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
On Tue, Mar 03, 2009 at 10:12:22PM +, Steve McIntyre wrote: I'm looking at my local mirror (slowly) update at the moment, and I've got to wondering: are the large -dbg packages actually really useful to anybody? I can't imagine that more than a handful of users ever install (to pick an example) the amarok-dbg packages, but we have multiple copies of a 70MB-plus .deb taking up mirror space and bandwidth. I can understand this for library packages, maybe, but for applications? There are people working on ways of compressing the debuginfo information, and I've been told they might have results within a couple of months. Part of the problem is that depending on how the package is built, the -dbg packages can be huge, so it makes the cost/benefit ratio somewhat painful. If the -dbg files were more like these sizes: 224 e2fslibs-dbg_1.41.3-1_i386.deb 52 libss2-dbg_1.41.3-1_i386.deb 452 e2fsprogs-dbg_1.41.3-1_i386.deb48 libuuid1-dbg_1.41.3-1_i386.deb 76 libblkid1-dbg_1.41.3-1_i386.deb48 uuid-runtime-dbg_1.41.3-1_i386.deb 44 libcomerr2-dbg_1.41.3-1_i386.deb I doubt there's be too much concern - Ted -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
On Tue, Mar 03, 2009 at 10:25:06PM -0500, Theodore Tso wrote: I'm looking at my local mirror (slowly) update at the moment, and I've got to wondering: are the large -dbg packages actually really useful to anybody? I can't imagine that more than a handful of users ever install (to pick an example) the amarok-dbg packages, but we have multiple copies of a 70MB-plus .deb taking up mirror space and bandwidth. I can understand this for library packages, maybe, but for applications? There are people working on ways of compressing the debuginfo information, and I've been told they might have results within a couple of months. Part of the problem is that depending on how the package is built, the -dbg packages can be huge, so it makes the cost/benefit ratio somewhat painful. If the -dbg files were more like these sizes: 224 e2fslibs-dbg_1.41.3-1_i386.deb 52 libss2-dbg_1.41.3-1_i386.deb 452 e2fsprogs-dbg_1.41.3-1_i386.deb48 libuuid1-dbg_1.41.3-1_i386.deb 76 libblkid1-dbg_1.41.3-1_i386.deb48 uuid-runtime-dbg_1.41.3-1_i386.deb 44 libcomerr2-dbg_1.41.3-1_i386.deb I doubt there's be too much concern Remaining concerns: - each of these dbg packages requires manual modification to the source package (incl. adding the package to debian/control) - each has to go through the NEW queue - each takes up space afterwards in the Packages file Much better if these can be generated centrally as part of the builds. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: -dbg packages; are they actually useful?
On Wed, Mar 4, 2009 at 2:17 PM, Steve Langasek vor...@debian.org wrote: If the -dbg files were more like these sizes: ... I doubt there's be too much concern Remaining concerns: - each of these dbg packages requires manual modification to the source package (incl. adding the package to debian/control) - each has to go through the NEW queue - each takes up space afterwards in the Packages file Much better if these can be generated centrally as part of the builds. Agreed. To be more useful, this would require that the ftpmaster team allow source-only uploads or at least always rebuild binary packages provided by maintainers on the buildds (where the build process can be more easily controlled). ftpmasters, what is your current position on source-only uploads or throwing away maintainer-built debs? -- bye, pabs http://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
