Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Monday 20 April 2015 12:46 PM, David Kalnischkies wrote: That can be very quickly quite a set of packages. apt ~23, apititude ~40, mpv (similar to mplayer) ~159, kate (KDEs notepad) ~465. [0] That can be tuned by excluding non-libraries, but that has its own drawbacks (private libraries shared between a very closely related set of packages for example), aka: For a quickshot direct dependencies are probably enough (personal observation; the times I needed debug symbols for non-direct dependencies are far and in between, but maybe I am just lucky). If you wanna go fullcircle, its probably better to analyse a core-dump for which symbols are needed exactly instead of getting everything. I think Ubuntu has a tool dubbed apport-retrace (Debian has it in experimental only) which is supposed to do that (but I just remember hearing the name in this context, nothing more) Just FYI. Apport is included in this year's GSoC, for Debian. There are a bunch of Debian specific features we'd like to see. Once those are done, we should be in a better position to propose apport for testing/unstable. And apport will be one of the prime consumers of these debug symbols. So thank you for reviving on this subject. -- Given the large number of mailing lists I follow, I request you to CC me in replies for quicker response signature.asc Description: OpenPGP digital signature
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On 2015-05-03 18:58, Guillem Jover wrote: Hi! On Tue, 2015-04-07 at 22:11:18 +0200, Niels Thykier wrote: A) Use .deb (i.e. the regular extension) with a new section. Is there any problem with using the existing debug section? Or is the different section used to distinguish that these are autogenerated perhaps? I picked debugsym to ensure it distinguished the automatically generated debug debs from hand-crafted ones while testing it. I am personally slightly in favour of keeping this as an (extra) discriminator. That said, I will not object to using the normal debug section. B) Use .ddeb (i.e. with an extra d). What where the motivations for using a different extension? I have heard of/can think of two reasons: * it was to have a trivial discriminator prior to unpacking / extracting information from the deb. * backwards compatibility with Ubuntu's setup. I can see the motivation for .udeb:s as they need to live in a different namespace, but that does not seem to be the case for debug debs. Please keep in mind that there /is/ a desire to keep ddebs trivially distinguishable from regular debs. Among other things, to facilitate putting ddebs in a separate part of the mirror to avoid inflating the size of the metadata on the mirror (for users not using ddebs). Assuming that any issue with debug .deb:s is fixed in the tools, is there any remaining reason to use .ddeb:s? To my knowledge: No - dpkg-genchanges and dak are the only known tools blocking the use of .deb. To be honest, I am not sure about dak, as I have not tried uploading with an out-of-band deb. That said, dak will want to know about them to avoid an explosion in the NEW queue. On 2015-04-07 21:10, Niels Thykier wrote: Both have their own advantages and disadvantages. In particular: A) means that almost every existing tool will handle the debug debs like a regular deb (which it is) and will generally work perfectly out of the box. - There are a couple of exceptions, but we are limited to something like lintian and dpkg-genchanges. I'd happily look into making dpkg-genchanges allow this. Thanks. I tried dpkg from git, which now allows them with only a warning. I would certainly be interested in having a (long-term) solution that did not warn about these. - There will be tools that might want to handle them differently. Programs like dak and reprepro might want to (support) put(ting) them in a different part of their repositories. They could already do that by keying on the Section, no? Otherwise the filename is also unique enough to be keyed on («*-dbgsym_*»)? That is my understanding as well. Note that Ubuntu have a few manually generated *-dbgsym_* packages (e.g. for the linux kernel). It is my understanding that these are in spirit intended to be considered regular ddebs. Accordingly, there ought to be no issue in filtering based on that file name pattern. [...] From my point of view, I am not strongly attached to one solution over the other: * I am slightly preferring A), but I am ready to implement either solution in the tools, I maintain. * The difference for debhelper is a single d and a section name. * The change for lintian is larger, but B) is the heavy solution and I already got a mostly working patch for that. Barring any strong reasoning behind B) above, I pretty much prefer A). Thanks, Guillem Great. :) To my knowledge, A) is also the solution that the FTP masters seemed to prefer. Thanks, ~Niels -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/55545904.2070...@thykier.net
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On 2015-04-19 19:10, David Kalnischkies wrote: On Sat, Apr 04, 2015 at 10:54:09AM +0200, Niels Thykier wrote: The resulting debs are installable with dpkg -i ( \o/ ). I have not tried anything fancy like setting up a local APT mirror and tried to convince APT do install it. I did and apt works with ddeb just fine, meaning it can happily print infos about them, download them and install them with dpkg. There are two exceptions as far as I can see: [... snip minor issues with .ddeb support in various APT tools ...] For reference, it seems like we will be picking the .deb route. So even these (minor) issues are unlikely to be a problem. So, super-cow approves (d)debs. \o/ (In fact, apt-dbg never became a thing as automatic ddebs were always very soon now in existence every time it came up. This time please…) I certainly intend to follow this all the way. And it especially approves the debhelper branch name. ;) I could imagine. : [...] btw: Is it planed to drop them into their own repository/component or are we gone blow up our regular Packages files with them? (you might guess from the wording which way I would prefer). Best regards David Kalnischkies It is my general understanding that most people would (strongly?) prefer that ddebs were placed in a separate component. I would be surprised if we did ended up including them in the current components. Thanks, ~Niels -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/55545af4.6030...@thykier.net
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On 2015-05-02 13:46, David Kalnischkies wrote: On Fri, May 01, 2015 at 11:46:42PM +0200, Niels Thykier wrote: […] ddeb support […] +1. \o/ - apt now properly handles the pkg:arch dependency. [...] I would revert the revert as this is potentially causing more trouble than the problems it is trying to solve (aka: I don't see why a debug package has to depend on the package it provides symbols for at all. If any the relation should be 'Enhances'…). Best regards David Kalnischkies I add the depends for the following reasons: * It is what we do with manual -dbg packages today and it is what people seem to expect. * It allows me to trivially deploy a doc-symlink to avoid an extra copy of the copyright file to create policy compliant debs. Now, IRT the pkg:arch dependency - it was to ensure that the you get the correct variant of your debug package. I can certainly appreciate that the (original?) Multi-arch spec does not support this for Multi-arch: foreign[1]. We have now ended up in a situation where people has made their own interpretation of how to handle this situation rending pkg:arch dependencies useless when pkg is multi-arch:foreign. It is what happens when people have to guess what something means. :) Thankfully, we got a solution that works perfectly for any other multi-arch value and foreign is just a minor inconvenience when APT guesses wrong on the (architecture for the) debug package. Thanks, ~Niels [1] As I recall, it does not really mention the pkg:arch dependencies. But it is a couple of years since I last read it, so I am quite possibly wrong here. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/55545ea6.5040...@thykier.net
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Sat, May 02, 2015 at 09:07:56AM -0400, James McCoy wrote: On Sat, May 02, 2015 at 01:46:25PM +0200, David Kalnischkies wrote: (aka: I don't see why a debug package has to depend on the package it provides symbols for at all. If any the relation should be 'Enhances'…). The intention is to ensure the debug symbols came from the same build as the binary packages which are being enhanced. Enhances doesn't provide that guarantee since it's a purely aesthetic relationship. The Why? Is something broken by the fact that you have a -dbg(sym) package installed, but not the package the debug symbols are for, or does anything break by having an outdated -dbg(sym) package – appart from your ability to make use of the symbols files? -data packages do not depend on their users even through they are useless without them. -doc packages do not depend on the things they document even through having an outdated version means stuff which is documented to work doesn't (which could be quite dangerous). Note also that the relation you are trying to express is stronger than can be expressed currently as Provides can satisfy such a relation (curtesy of versioned provides being supported by dpkg now), while they are hardly satisfying the intention. I think this Depends is only needed because most manual debug packages have it, so by induction, the automatic ones need it, too. Oh, and I would prefer teaching our packages managers to deal with enhances better, rather than declaring them purely aesthetic forever… There is nothing wrong with teaching them to keep installed (reverse) enhance relations satisfied, much like they do for recommends (and should for suggests) for example. Keeping debug packages autoinstalled, but tagged as non-garbage as long as the package they enhance is installed would be fun, too. There are countless more plans for great things. Unfortuently manpower is too limited to have them all done at once. I hope we can get at least some for stretch… Best regards David Kalnischkies signature.asc Description: Digital signature
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
Hi! On Tue, 2015-04-07 at 22:11:18 +0200, Niels Thykier wrote: A) Use .deb (i.e. the regular extension) with a new section. Is there any problem with using the existing debug section? Or is the different section used to distinguish that these are autogenerated perhaps? B) Use .ddeb (i.e. with an extra d). What where the motivations for using a different extension? I can see the motivation for .udeb:s as they need to live in a different namespace, but that does not seem to be the case for debug debs. Assuming that any issue with debug .deb:s is fixed in the tools, is there any remaining reason to use .ddeb:s? On 2015-04-07 21:10, Niels Thykier wrote: Both have their own advantages and disadvantages. In particular: A) means that almost every existing tool will handle the debug debs like a regular deb (which it is) and will generally work perfectly out of the box. - There are a couple of exceptions, but we are limited to something like lintian and dpkg-genchanges. I'd happily look into making dpkg-genchanges allow this. - There will be tools that might want to handle them differently. Programs like dak and reprepro might want to (support) put(ting) them in a different part of their repositories. They could already do that by keying on the Section, no? Otherwise the filename is also unique enough to be keyed on («*-dbgsym_*»)? - This is *currently not working* since dpkg-genchanges errors out on the auto-generated .deb files. See above. :) B) means that .ddebs can be special cased on filenames rather than on section (like udebs). Furthermore, there might be a lot of things that do not need to support .ddebs at all. Personally I think it breaks expectations, from muscle memory: # dpkg -i *.deb to many tools that might process debs and expect them to have that extension, which of course could be fixed but we have to take into account tools outside of the Debian archive. - Downside is that adding support is a manual extra step for many tools, that (besides the filename) would otherwise be able to handle .ddebs immediately. For example dpkg-scanpackages can be told to use a different package type, but it cannot be told to recognize multiple package types for the same invocation. I guess in this case you can cat the result, but still: $ ( dpkg-scanpackages . ; dpkg-scanpackages --type ddeb . ) Pacakges this would imply either fixing the tool or going around and changing lots of scripts for custom repositories. - On the plus side: dpkg-genchanges in Jessie can support this solution immediately with a minor warning. Sure, immediate deployment is a plus, but I'd rather we'd carefully consider the consequences of any such decission that might be pretty hard to reverse course in the future in case we find it to be too problematic. From my point of view, I am not strongly attached to one solution over the other: * I am slightly preferring A), but I am ready to implement either solution in the tools, I maintain. * The difference for debhelper is a single d and a section name. * The change for lintian is larger, but B) is the heavy solution and I already got a mostly working patch for that. Barring any strong reasoning behind B) above, I pretty much prefer A). Thanks, Guillem -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20150503165831.gb9...@gaara.hadrons.org
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Fri, May 01, 2015 at 11:46:42PM +0200, Niels Thykier wrote: […] ddeb support […] +1. \o/ - apt now properly handles the pkg:arch dependency. For different values of properly – apt isn't the only thing involved here, you have to consider the reaction of dpkg and dose as well and these 3 tools aren't agreeing on the interpretation of pkg:arch in combination with M-A:foreign and various other things at the moment: https://lists.alioth.debian.org/pipermail/multiarch-devel/2015-April/99.html (The initial problem, that you can install packages on single arch system where arch == native-arch which is what debhelper is trying to pull here was fixed, yes, but that doesn't mean that you will actually get a package from the native-arch. -- If anyone wants to comment this, don't do it here, do it in the thread I linked AFTER reading the thread. There is enough complexity and text to fill an afternoon; no need to send a comment in the lunch break with your smartphone…). I would revert the revert as this is potentially causing more trouble than the problems it is trying to solve (aka: I don't see why a debug package has to depend on the package it provides symbols for at all. If any the relation should be 'Enhances'…). Best regards David Kalnischkies signature.asc Description: Digital signature
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Sat, May 02, 2015 at 01:46:25PM +0200, David Kalnischkies wrote:
(aka: I don't see why a debug
package has to depend on the package it provides symbols for at all. If
any the relation should be 'Enhances'…).
The intention is to ensure the debug symbols came from the same build as
the binary packages which are being enhanced. Enhances doesn't provide
that guarantee since it's a purely aesthetic relationship. The
alternative is to have all the packages which are enhanced by the debug
symbols declare “Breaks: foo-dbg ( ${binary:Version}), foo-dbg (
${binary:Version})”
Cheers,
--
James
GPG Key: 4096R/331BA3DB 2011-12-05 James McCoy james...@debian.org
signature.asc
Description: Digital signature
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On 2015-04-04 10:54, Niels Thykier wrote: On 2015-04-04 09:54, Josselin Mouette wrote: Le jeudi 02 avril 2015 à 19:37 +0200, Esokrates a écrit : Hi, I am particularly interested in automatic debug packages, as the current situation is pretty messy imho. I found https://wiki.debian.org/AutomaticDebugPackages. Does anyone know the status of this? Will this be a goal for Stretch? This and reproducible builds would make Debian the perfect distribution for me. Whats in the way of making it happen? Lack of people willing to do it? Last time I checked, dak was still missing code to handle the generated .ddeb files. Cheers, And it *still* does! But there are a few things that have changed! * There is an experimental branch for debhelper to generate these automatically available. [...] Enjoy, ~Niels [...] This branch (with some bug fixes) are now in experimental if anyone wants to play with it. * Remember to set DH_BUILD_DDEBS! Else ddebs will not be built * dak support is still missing * Thanks to the reproducibility team, the APT maintainers and the dpkg maintainers. They have either helped me solve a few bugs or fixed a bug in their respective tools. In particular: - the reproducibility team reported a number of brown paper bag bugs (like dh_strip throwing away debug symbols in regular -dbg pkgs /o\ ). - dpkg-genchanges no longer emits warnings when building .ddebs. - apt now properly handles the pkg:arch dependency. * Some caveats remain (like lack of documentation and there is no valid migration path from -dbg to .ddebs). Thanks, ~Niels -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/5543f442.2010...@thykier.net
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Mon, Apr 20, 2015 at 09:50:00AM +0800, Paul Wise wrote: On Mon, Apr 20, 2015 at 1:10 AM, David Kalnischkies wrote: I would presume most derivatives aren't using it either Most derivatives appear to use reprepro but there is one using apt-ftparchive https://wiki.debian.org/Derivatives/CensusFull I knew it. I had a look at the census page especially as the list had a discussion about it recently, but couldn't find the field there. Not all derivatives seem to declare it and I looked at the wrong ones… I know that a few are using it which haven't declared it. Ubuntu e.g. does (at least that is what I figure from the bugreports). The point was mainly: apt-ftparchive/jessie doesn't support it out of the box, but its easy to change that if someone would need it. (I have my doubts that all others do support it out of the box either) https://wiki.debian.org/Derivatives/Census/Lihuen That is an interesting. The script they use does use the manual steps 'packages' and so on, it does mention 'generate' in a comment through. Gonna talk to them (and d-derivatives@ at large) later. They aren't supporting InRelease nor .xz for example which would be nice if they do. I think there will be some work upon us to make ddebs supported well (I invision something like a apt-get debugsymbols foo which installs the package foo-dbgsym and maybe optionally also the debug packages of the direct dependencies libfoo1 (libfoo-dbgsym) and libbar0.1 (python3-bar-dbgsym as it is the c-binding of a python library as you might (not) have guessed).) but lets get them first, shall we? :) As a user of debug packages I'd like installing foo-dbg to pull in all the -dbg packages I would need to dump a backtrace from GDB. So basically all recursive reverse dependencies. That can be very quickly quite a set of packages. apt ~23, apititude ~40, mpv (similar to mplayer) ~159, kate (KDEs notepad) ~465. [0] That can be tuned by excluding non-libraries, but that has its own drawbacks (private libraries shared between a very closely related set of packages for example), aka: For a quickshot direct dependencies are probably enough (personal observation; the times I needed debug symbols for non-direct dependencies are far and in between, but maybe I am just lucky). If you wanna go fullcircle, its probably better to analyse a core-dump for which symbols are needed exactly instead of getting everything. I think Ubuntu has a tool dubbed apport-retrace (Debian has it in experimental only) which is supposed to do that (but I just remember hearing the name in this context, nothing more) [0] very hacky approximation with apt-cache depends apt --important --no-conflicts --no-replaces --no-breaks --recurse -o APT::Architectures=amd64 -o APT::Cache::ShowOnlyFirstOr=1 | grep -v '^ ' | wc -l Best regards David Kalnischkies signature.asc Description: Digital signature
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Mon, Apr 20, 2015 at 1:10 AM, David Kalnischkies wrote: I would presume most derivatives aren't using it either Most derivatives appear to use reprepro but there is one using apt-ftparchive https://wiki.debian.org/Derivatives/CensusFull https://wiki.debian.org/Derivatives/Census/Lihuen I think there will be some work upon us to make ddebs supported well (I invision something like a apt-get debugsymbols foo which installs the package foo-dbgsym and maybe optionally also the debug packages of the direct dependencies libfoo1 (libfoo-dbgsym) and libbar0.1 (python3-bar-dbgsym as it is the c-binding of a python library as you might (not) have guessed).) but lets get them first, shall we? :) As a user of debug packages I'd like installing foo-dbg to pull in all the -dbg packages I would need to dump a backtrace from GDB. So basically all recursive reverse dependencies. btw: Is it planed to drop them into their own repository/component or are we gone blow up our regular Packages files with them? (you might guess from the wording which way I would prefer). IIRC it was always planned to put them in a separate place. -- bye, pabs https://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CAKTje6GFzSJCyaFwXz06gyUtQc0=5__=1_371_yrdwvm3qg...@mail.gmail.com
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Sat, Apr 04, 2015 at 10:54:09AM +0200, Niels Thykier wrote: The resulting debs are installable with dpkg -i ( \o/ ). I have not tried anything fancy like setting up a local APT mirror and tried to convince APT do install it. I did and apt works with ddeb just fine, meaning it can happily print infos about them, download them and install them with dpkg. There are two exceptions as far as I can see: - apt-ftparchive doesn't know about '.ddeb', so by default neither Contents nor Packages files created by it contain information about them. Users of apt-ftparchive contents/packages are (surprisingly) out of luck as far as I can see and have to wait for a patch (= 2 lines), but users of apt-ftparchive generate can configure the used extension, see apt-ftparchive manpage on how to do that. Debian uses dak to create the archive, so that isn't a problem per-se. I would presume most derivatives aren't using it either as alternatives are plenty. Those who do are very likely using generate as its just strictly better than manually creating Packages files with 'apt-ftparchive packages'. I guess most repository creators have this or similar problems and solutions through and that doesn't seem like all to important for the adoption. - apt/experimental supports installing local .deb files. That doesn't support the '.ddeb' extension yet. I leave it up to the reader to figure out how much of a dealbreaker that is… So, super-cow approves (d)debs. (In fact, apt-dbg never became a thing as automatic ddebs were always very soon now in existence every time it came up. This time please…) And it especially approves the debhelper branch name. ;) I think there will be some work upon us to make ddebs supported well (I invision something like a apt-get debugsymbols foo which installs the package foo-dbgsym and maybe optionally also the debug packages of the direct dependencies libfoo1 (libfoo-dbgsym) and libbar0.1 (python3-bar-dbgsym as it is the c-binding of a python library as you might (not) have guessed).) but lets get them first, shall we? :) btw: Is it planed to drop them into their own repository/component or are we gone blow up our regular Packages files with them? (you might guess from the wording which way I would prefer). Best regards David Kalnischkies signature.asc Description: Digital signature
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On 2015-04-09 09:25, Esokrates wrote: On Tuesday, April 07, 2015 10:11:18 PM Niels Thykier wrote: [...] So mostly that is more a decision making (political) problem, than a technical one. It is not entirely clear to me that we any have (major) political issues IRT ddebs. People generally seem to be positive (or worst case indifferent) to the idea itself. There also seem to be agreement about what goes into the ddebs, where it is placed and they are just another deb (etc.). As mentioned, the only real concern seem to be whether or not it should be a .ddeb or .deb. Though I doubt it is going to be problematic - if this was truly something people felt for strongly, I am certain they would have expressed their opinion by now. Stretch is a two year time frame though, which makes me kinda sad. Thanks for you effort though, keep up the amazing work! If I understand correctly, if it would have been something for stretch, either A or B would have been decided already and partly implemented, right? I believe we can implement this for Stretch if we wanted to. Unlike multi-arch, build-profiles (etc.), ddebs are already supported by our (14-days-from-now-new) stable release. Namely, * All the groundwork requirements (pkg:arch dependencies etc.) are already covered by previous efforts. - I.e. everything that would require a full-cycle for support are already done in time for Jessi * What we are missing is mostly support from our infrastructure and repository tools. As far as I am informed, the main blockers here are: - With .ddebs = dak, debhelper - With .debs = dak, debhelper, dpkg-dev None of which * Known list of tools where support is (very) nice-to-have, but not a strict blocker. - lintian - though people on d-mentors might disagree. ;) - reprepro and similar - others? * List of (assumed) support is completely optional or even unnecessary - buildd/sbuild/wanna-build - it should just be a regular build for these with debhelper doing the heavy lifting. - britney - the ddebs build by debhelper are just as (co-) installable as the .deb they are based on. I am looking forward to the day, when both reproducible builds and automatic debug package exist, that will be an awesome future! Thanks very much for outlining this, Niels! I certainly agree! I cannot take credit for the reproducible builds though. :) [Different mail, same sender]: Or maybe another question: Will it be ready for sid, soon? Sorry, I am really excited about this. Good question really. I can certainly upload a version of debhelper to unstable that has /opt-in/ support for ddebs when Stretch opens. However, to have them on the mirror largely depends on when dak gets support for ddebs. Unfortunately, I do not have any ETA on that. Thanks, ~Niels -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/5528adb6.8050...@thykier.net
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Tuesday, April 07, 2015 10:11:18 PM Niels Thykier wrote: On 2015-04-07 21:10, Niels Thykier wrote: On 2015-04-04 12:58, Esokrates wrote: On Saturday, April 04, 2015 10:54:09 AM Niels Thykier wrote: [...] I know predictions are hard, but is there a plan to get things done for the next release (Stretch)? At this point, there is no plan, sorry. However, we got a functional prototype (for part of the problem) and some people poking a bit at it from a design view. I received conflicting remarks: A) Use .ddeb (i.e. with an extra d). B) Use .deb (i.e. the regular extension) with a new section. I managed to confuse myself here and swapped A and B in the above. What I meant to write was: A) Use .deb (i.e. the regular extension) with a new section. B) Use .ddeb (i.e. with an extra d). The rest should now make sense - apologies for the confusion: Both have their own advantages and disadvantages. In particular: A) means that almost every existing tool will handle the debug debs like a regular deb (which it is) and will generally work perfectly out of the box. - There are a couple of exceptions, but we are limited to something like lintian and dpkg-genchanges. - There will be tools that might want to handle them differently. Programs like dak and reprepro might want to (support) put(ting) them in a different part of their repositories. - This is *currently not working* since dpkg-genchanges errors out on the auto-generated .deb files. B) means that .ddebs can be special cased on filenames rather than on section (like udebs). Furthermore, there might be a lot of things that do not need to support .ddebs at all. - Downside is that adding support is a manual extra step for many tools, that (besides the filename) would otherwise be able to handle .ddebs immediately. - On the plus side: dpkg-genchanges in Jessie can support this solution immediately with a minor warning. From my point of view, I am not strongly attached to one solution over the other: * I am slightly preferring A), but I am ready to implement either solution in the tools, I maintain. * The difference for debhelper is a single d and a section name. * The change for lintian is larger, but B) is the heavy solution and I already got a mostly working patch for that. [...] So mostly that is more a decision making (political) problem, than a technical one. Stretch is a two year time frame though, which makes me kinda sad. Thanks for you effort though, keep up the amazing work! If I understand correctly, if it would have been something for stretch, either A or B would have been decided already and partly implemented, right? I am looking forward to the day, when both reproducible builds and automatic debug package exist, that will be an awesome future! Thanks very much for outlining this, Niels! -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/3487072.50TJyGfKk4@ubuntu
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Thursday, April 09, 2015 09:25:44 AM Esokrates wrote: So mostly that is more a decision making (political) problem, than a technical one. Stretch is a two year time frame though, which makes me kinda sad. Thanks for you effort though, keep up the amazing work! If I understand correctly, if it would have been something for stretch, either A or B would have been decided already and partly implemented, right? I am looking forward to the day, when both reproducible builds and automatic debug package exist, that will be an awesome future! Thanks very much for outlining this, Niels! Or maybe another question: Will it be ready for sid, soon? Sorry, I am really excited about this. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/3804029.DFJoepXrID@ubuntu
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On 2015-04-04 12:58, Esokrates wrote: On Saturday, April 04, 2015 10:54:09 AM Niels Thykier wrote: [...] - Trying to get the reproducible team to try it out to see if it regresses anything (incl. reproducible builds) I guess the ddeb's are meant to be reproducible too? Yes. The .ddebs are currently being built by the reproducibility team on jenkins.debian.net and they are indeed reproducible themselves (after we extended a patch the reproducible debhelper patch series to account for them IRT mtimes). - It *does* cause dpkg-genchanges to emit warnings about uninitialized values (I think 3 per ddeb). Related to #781074 - DOES *NOT* PRODUCE DDEBS FOR PACKAGES COVERED BY AN EXISTING -dbg! Sounds like you plan to keep -dbg packages? Or just for migration? Ubuntu kept -dbg packages and tries to maintain -dbgsym along it (but they only cover packages that no -dbg package exists for, otherwise it will be an empty package, no idea if they situation is still like that there ...), however the situation was kind of messy, I often got file conflicts. Things were not consistent at all last time I needed debug packages for a lot of stuff there. Simply put: If you installed the debug package for every installed package, you get the hell of a mess, this was almost undoable. (That test maybe does not make a lot of sense, but it did show that things were not optimal.) I have no particular interest in keeping existing manual -dbg packages. However, there seemed to be little point in creating a .ddeb which is a perfect clone of an existing -dbg with neither of them being co-installable. Please also note that there are some existing manual debug packages that we *cannot* migrate to ddeb currently (at least not as-is). E.g. perl-debug provides debug symbols but also a debugperl binary etc. So the prototype patch plays it safe and only generate .ddebs if the debug symbols would otherwise have been discarded. This also means that if the -dbg does not cover all binaries, debhelper will generate .ddebs for the uncovered binaries (where applicable). - DOES *NOT* ADD BREAKS/REPLACES FOR MIGRATING FROM -dbg TO .ddeb! For reference, this limitation is due to time constraints - I intend to have a solution for this problem (but it will probably have to be manual - maybe an argument for dh_gencontrol). [...] I know predictions are hard, but is there a plan to get things done for the next release (Stretch)? At this point, there is no plan, sorry. However, we got a functional prototype (for part of the problem) and some people poking a bit at it from a design view. I received conflicting remarks: A) Use .ddeb (i.e. with an extra d). B) Use .deb (i.e. the regular extension) with a new section. Both have their own advantages and disadvantages. In particular: A) means that almost every existing tool will handle the debug debs like a regular deb (which it is) and will generally work perfectly out of the box. - There are a couple of exceptions, but we are limited to something like lintian and dpkg-genchanges. - There will be tools that might want to handle them differently. Programs like dak and reprepro might want to (support) put(ting) them in a different part of their repositories. - This is *currently not working* since dpkg-genchanges errors out on the auto-generated .deb files. B) means that .ddebs can be special cased on filenames rather than on section (like udebs). Furthermore, there might be a lot of things that do not need to support .ddebs at all. - Downside is that adding support is a manual extra step for many tools, that (besides the filename) would otherwise be able to handle .ddebs immediately. - On the plus side: dpkg-genchanges in Jessie can support this solution immediately with a minor warning. From my point of view, I am not strongly attached to one solution over the other: * I am slightly preferring A), but I am ready to implement either solution in the tools, I maintain. * The difference for debhelper is a single d and a section name. * The change for lintian is larger, but B) is the heavy solution and I already got a mostly working patch for that. Thanks for the very comprehensive status update and your awesome work! You are welcome. :) Thanks, ~Niels -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/55242b97.5080...@thykier.net
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On 2015-04-07 21:10, Niels Thykier wrote: On 2015-04-04 12:58, Esokrates wrote: On Saturday, April 04, 2015 10:54:09 AM Niels Thykier wrote: [...] I know predictions are hard, but is there a plan to get things done for the next release (Stretch)? At this point, there is no plan, sorry. However, we got a functional prototype (for part of the problem) and some people poking a bit at it from a design view. I received conflicting remarks: A) Use .ddeb (i.e. with an extra d). B) Use .deb (i.e. the regular extension) with a new section. I managed to confuse myself here and swapped A and B in the above. What I meant to write was: A) Use .deb (i.e. the regular extension) with a new section. B) Use .ddeb (i.e. with an extra d). The rest should now make sense - apologies for the confusion: Both have their own advantages and disadvantages. In particular: A) means that almost every existing tool will handle the debug debs like a regular deb (which it is) and will generally work perfectly out of the box. - There are a couple of exceptions, but we are limited to something like lintian and dpkg-genchanges. - There will be tools that might want to handle them differently. Programs like dak and reprepro might want to (support) put(ting) them in a different part of their repositories. - This is *currently not working* since dpkg-genchanges errors out on the auto-generated .deb files. B) means that .ddebs can be special cased on filenames rather than on section (like udebs). Furthermore, there might be a lot of things that do not need to support .ddebs at all. - Downside is that adding support is a manual extra step for many tools, that (besides the filename) would otherwise be able to handle .ddebs immediately. - On the plus side: dpkg-genchanges in Jessie can support this solution immediately with a minor warning. From my point of view, I am not strongly attached to one solution over the other: * I am slightly preferring A), but I am ready to implement either solution in the tools, I maintain. * The difference for debhelper is a single d and a section name. * The change for lintian is larger, but B) is the heavy solution and I already got a mostly working patch for that. [...] -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/552439e6.40...@thykier.net
Re: Experimental ddeb support in debhelper and lintian (Was: Re: -dbg packages; are they actually useful?)
On Saturday, April 04, 2015 10:54:09 AM Niels Thykier wrote: Last time I checked, dak was still missing code to handle the generated .ddeb files. Cheers, And it *still* does! But there are a few things that have changed! * There is an experimental branch for debhelper to generate these automatically available. - Requires a export DH_BUILD_DDEBS=1 to trigger the code path - It applies to *all* compat levels. - Trying to get the reproducible team to try it out to see if it regresses anything (incl. reproducible builds) I guess the ddeb's are meant to be reproducible too? - It *does* cause dpkg-genchanges to emit warnings about uninitialized values (I think 3 per ddeb). Related to #781074 - DOES *NOT* PRODUCE DDEBS FOR PACKAGES COVERED BY AN EXISTING -dbg! Sounds like you plan to keep -dbg packages? Or just for migration? Ubuntu kept -dbg packages and tries to maintain -dbgsym along it (but they only cover packages that no -dbg package exists for, otherwise it will be an empty package, no idea if they situation is still like that there ...), however the situation was kind of messy, I often got file conflicts. Things were not consistent at all last time I needed debug packages for a lot of stuff there. Simply put: If you installed the debug package for every installed package, you get the hell of a mess, this was almost undoable. (That test maybe does not make a lot of sense, but it did show that things were not optimal.) - DOES *NOT* ADD BREAKS/REPLACES FOR MIGRATING FROM -dbg TO .ddeb! - Branch at [DH-BRANCH] * Half-done branch for lintian to actually look at ddebs - Known false-positive triggered for ddebs (output says FIXME, so it should be trivial to spot). - Branch at [LINT-BRANCH] The resulting debs are installable with dpkg -i ( \o/ ). I have not tried anything fancy like setting up a local APT mirror and tried to convince APT do install it. *Known to be missing*: * Documentation in debhelper (incl. changelog entries) * A solution to the -dbg = .ddeb migration, so you can do a -2 upload that uses .ddeb instead of -dbg and *not* get file conflicts. * FTP master approval + dak support - Possibly a solution that does not double the size of Packages. * APT support or verification that it works out of the box - help welcome. * dpkg (dpkg-genchanges) should pref. not emit uninitialised warnings - Probably related #781074. My guess is that a call to debarch_eq is the source of all warnings. I have not pinpointed which call. ... and I am late for something now, so I will cut it short here. Enjoy, ~Niels [DH-BRANCH]: https://anonscm.debian.org/cgit/users/nthykier/debhelper.git/log/?h=super-co w-ddebs-support [LINT-BRANCH]: http://anonscm.debian.org/cgit/users/nthykier/lintian.git/log/?h=ddeb-suppor t I know predictions are hard, but is there a plan to get things done for the next release (Stretch)? Thanks for the very comprehensive status update and your awesome work! -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/2478425.nmYp5U8jSL@ubuntu
