Re: Proposed new release goal: Dependency/file list predictability
Bill Allombert [EMAIL PROTECTED] wrote: This should be fixed by rebuilding af. However, since we are moving to a new menu structure, the menu file will need to be updated[1], so the packages will need a new sourceful upload anyway. The footnote was missing, unfortunately: I'm really curious about the plans for the menu structure update. Regards, Frank -- Frank Küster Single Molecule Spectroscopy, Protein Folding @ Inst. f. Biochemie, Univ. Zürich Debian Developer (teTeX/TeXLive)
Re: Proposed new release goal: Dependency/file list predictability
On Mon, Jun 25, 2007 at 08:22:17AM +0200, Frank Küster wrote: Bill Allombert [EMAIL PROTECTED] wrote: This should be fixed by rebuilding af. However, since we are moving to a new menu structure, the menu file will need to be updated[1], so the packages will need a new sourceful upload anyway. The footnote was missing, unfortunately: I'm really curious about the plans for the menu structure update. The plan is explained in bug #361418. If you to comment on it, please do it ASAP. The footnote was supposed to refer to the fact that menu will silently translate the old structure to the new one for a large number of menu entries, but unfortunately not for af, so I discarded the footnote. Cheers, -- Bill. [EMAIL PROTECTED] Imagine a large red swirl here. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Proposed new release goal: Dependency/file list predictability
Bill Allombert [EMAIL PROTECTED] wrote: On Mon, Jun 25, 2007 at 08:22:17AM +0200, Frank Küster wrote: Bill Allombert [EMAIL PROTECTED] wrote: This should be fixed by rebuilding af. However, since we are moving to a new menu structure, the menu file will need to be updated[1], so the packages will need a new sourceful upload anyway. The footnote was missing, unfortunately: I'm really curious about the plans for the menu structure update. The plan is explained in bug #361418. If you to comment on it, please do it ASAP. The footnote was supposed to refer to the fact that menu will silently translate the old structure to the new one for a large number of menu entries, but unfortunately not for af, so I discarded the footnote. Ah, okay - I knew about the automatic translation, but was under the impression that this should work for all or at least nearly all packages. Regards, Frank -- Frank Küster Single Molecule Spectroscopy, Protein Folding @ Inst. f. Biochemie, Univ. Zürich Debian Developer (teTeX/TeXLive)
Re: Proposed new release goal: Dependency/file list predictability
On 24/06/07 at 20:25 +0200, Steinar H. Gunderson wrote: Steinar, do you want to merge your release goal with that one, or do you prefer me to file a seperate release goal? The main reason why I think they should be merged is that the way to detect issues is similar. I'd prefer to have it as a separate release goal, or at least a different usertag (which is really all that matters; release goals are not binary achieved/not achieved anyway). Ok, I'm fine with that. This release goal (fresh build doesn't differ from the archive) won't probably result in bugs being filed, since, in most cases, it is enough to binNMU the affected packages. -- | Lucas Nussbaum | [EMAIL PROTECTED] http://www.lucas-nussbaum.net/ | | jabber: [EMAIL PROTECTED] GPG: 1024D/023B3F4F | -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Proposed new release goal: Dependency/file list predictability
On Mon, Jun 25, 2007 at 12:34:30PM +0200, Frank Küster wrote: Bill Allombert [EMAIL PROTECTED] wrote: On Mon, Jun 25, 2007 at 08:22:17AM +0200, Frank Küster wrote: Bill Allombert [EMAIL PROTECTED] wrote: This should be fixed by rebuilding af. However, since we are moving to a new menu structure, the menu file will need to be updated[1], so the packages will need a new sourceful upload anyway. The footnote was missing, unfortunately: I'm really curious about the plans for the menu structure update. The plan is explained in bug #361418. If you to comment on it, please do it ASAP. The footnote was supposed to refer to the fact that menu will silently translate the old structure to the new one for a large number of menu entries, but unfortunately not for af, so I discarded the footnote. Ah, okay - I knew about the automatic translation, but was under the impression that this should work for all or at least nearly all packages. Well it can work for all the packages, but there is no direct mapping from the old structure to the new one, and special-casing two hundred packages (out of 2500 providing menu entry) is something I would really much avoid. Much better to NMU two hundred packages to fix the menu entries properly. Some packages deserve a sourceful upload this millenium... Cheers, -- Bill. [EMAIL PROTECTED] Imagine a large red swirl here. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Proposed new release goal: Dependency/file list predictability
On Fri, Jun 22, 2007 at 09:23:16AM +0200, Lucas Nussbaum wrote: However, with this setup, you only check that building packages in non-clean environments doesn't significantly affect the package. It would be interesting to check as well if the resulting package matches what is in the archive, to some point. I already do that, but mainly as an optimization at this point. I do agree with your goal, though, but it's very hard to determine what's a bug and what's not. This package hasn't transitioned to newer libfoo yet is, IMO, not a bug in itself. BTW, I've built everything in optional up to the letter c (plus all of required, standard and important), and so far, 28 bugs have showed up. (Also, 44 FTBFS bugs have showed up, but I haven't filtered out those that FTBFS in clean chroots yet, and I guess those would account for nearly all.) Extrapolating from that, it would seem that slightly over 200 bugs of this class exist in the archive. I only started the full build on hydra (thanks, Joey) yesterday, so I assume I'll have a first approximation of the number of bugs in a couple of days. - some files will differ, because of: + date/time information + newer compiler versions causing better optimization + ... Yes, diffing binary files properly is very hard. So, I'd like to extend this release goal to something like binary packages predictability (to some extend). This would mostly result in a lot of binNMUs to update the binary packages to the current state of unstable, so in most cases, it shouldn't put a lot of load on maintainers. The number of issues is unknown ; it depends on how close we want packages to match. Do you think it's a good idea? I'm unsure. I do like the idea, but I'm concerned it would be difficult to determine what's an acceptable difference and what is not. Steinar, do you want to merge your release goal with that one, or do you prefer me to file a seperate release goal? The main reason why I think they should be merged is that the way to detect issues is similar. I'd prefer to have it as a separate release goal, or at least a different usertag (which is really all that matters; release goals are not binary achieved/not achieved anyway). /* Steinar */ -- Homepage: http://www.sesse.net/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Proposed new release goal: Dependency/file list predictability
Matthew Johnson wrote: Also, you should consider build systems which switch using the alternatives system. Debian Java policy says that debian/rules must specify the build system to use explicitly, but there are a number of packages which don't. If you know of any such packages, please file bug reports. That should really be fixed. Marcus -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Proposed new release goal: Dependency/file list predictability
On Fri, Jun 22, 2007 at 11:25:44PM +0100, Enrico Zini wrote: - compute a delicateness index of a package by summing the SAT temperature of all packages that depend on it, and then give special attention to the most delicate packages to see RC bugs, maintainer activity and so on. I'm working on creating a sample data, but I'll be away for the week-end. I managed to hack it together just in time: a sample of the 50 most 'delicate' packages is in: http://people.debian.org/~enrico/2007-06/sample-reverse-sat.txt Ciao, Enrico -- GPG key: 1024D/797EBFAB 2000-12-05 Enrico Zini [EMAIL PROTECTED] signature.asc Description: Digital signature
Re: Proposed new release goal: Dependency/file list predictability
On Sat, Jun 23, 2007 at 09:35:25AM +0100, Enrico Zini wrote: On Fri, Jun 22, 2007 at 11:25:44PM +0100, Enrico Zini wrote: - compute a delicateness index of a package by summing the SAT temperature of all packages that depend on it, and then give special attention to the most delicate packages to see RC bugs, maintainer activity and so on. I'm working on creating a sample data, but I'll be away for the week-end. I managed to hack it together just in time: a sample of the 50 most 'delicate' packages is in: http://people.debian.org/~enrico/2007-06/sample-reverse-sat.txt wow, this looks pretty accurate, though adduser seems misplaced in that list. You should remove arch:all packages, they are seldomly a concern for migrations. -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgpftyAvaZK4P.pgp Description: PGP signature
Re: Proposed new release goal: Dependency/file list predictability
On Fri, Jun 22, 2007 at 09:37:39PM +0200, Lucas Nussbaum wrote: af: files list differ: -./usr/lib/menu -./usr/lib/menu/af +./usr/share/menu +./usr/share/menu/af Non-ancient debhelper. Still, what do we want to do about those? This should be fixed by rebuilding af. However, since we are moving to a new menu structure, the menu file will need to be updated[1], so the packages will need a new sourceful upload anyway. Cheers, -- Bill. [EMAIL PROTECTED] Imagine a large red swirl here. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Proposed new release goal: Dependency/file list predictability
As discussed during DebConf, I'd like to propose a new release goal: Packages should not only build in clean chroots, but also in non-clean environments. Specifically, adding extra packages from the archive into the build environment (that are not in Build-Conflicts) should not affect the resulting package in any noticeable way. Also, you should consider build systems which switch using the alternatives system. Debian Java policy says that debian/rules must specify the build system to use explicitly, but there are a number of packages which don't. These will build if only their build-depends are installed or if their build system is the current alternative, but not if they were installed in a different order. Matt -- Matthew Johnson signature.asc Description: Digital signature
Re: Proposed new release goal: Dependency/file list predictability
On 22/06/07 at 01:49 +0200, Steinar H. Gunderson wrote: As discussed during DebConf, I'd like to propose a new release goal: Packages should not only build in clean chroots, but also in non-clean environments. Specifically, adding extra packages from the archive into the build environment (that are not in Build-Conflicts) should not affect the resulting package in any noticeable way. To this end, I've set up the build daemon from Hell (BDFH) on my machine, currently doing script testing. (Joey Hess has kindly promised to donate CPU time on a four-CPU machine so we can push through the entire archive at reasonable speeds at regular intervals; the setup will be moved there as soon as it's stable.) The idea is to build a package both in a pbuilder and in a really filled chroot -- it currently contains 18GB of packages, which is most of the devel and libdevel sections. What is compared is: - The list of Depends. - The list of Recommends. - The list of filenames. I really like the idea. However, with this setup, you only check that building packages in non-clean environments doesn't significantly affect the package. It would be interesting to check as well if the resulting package matches what is in the archive, to some point. Obviously, packages can't match perfectly: - binaries should depend on the same package, but could depend on different versions (even if Raphael's work will probably mitigate this) - some files will differ, because of: + date/time information + newer compiler versions causing better optimization + ... In january, I worked on a script that compares two build results (both sources and binaries packages), and compared the content of the archive with the result of one of my rebuilds. The differences were quite huge. Some examples (back from January of course): acepack: r-cran-acepack's depends differ: -Depends: libc6 , libg2c0 , libgcc1 , r-base-core +Depends: libc6 , libgcc1 , libgfortran1 , r-base-core alamin: alamin-client's postinst differs: if [ -x /etc/init.d/alamin-server ]; then update-rc.d alamin-server defaults /dev/null if [ -x `which invoke-rc.d 2/dev/null` ]; then - invoke-rc.d alamin-server start || exit 0 + invoke-rc.d alamin-server start || exit $? else - /etc/init.d/alamin-server start || exit 0 + /etc/init.d/alamin-server start || exit $? fi fi ada-mode: files list differ: -./usr/share/doc/ada-mode/html/Using-non-standard-file-names.html +./usr/share/doc/ada-mode/html/Using-non_002dstandard-file-names.html af: files list differ: -./usr/lib/menu -./usr/lib/menu/af +./usr/share/menu +./usr/share/menu/af etc. (you get the idea) So, I'd like to extend this release goal to something like binary packages predictability (to some extend). This would mostly result in a lot of binNMUs to update the binary packages to the current state of unstable, so in most cases, it shouldn't put a lot of load on maintainers. The number of issues is unknown ; it depends on how close we want packages to match. Do you think it's a good idea? Steinar, do you want to merge your release goal with that one, or do you prefer me to file a seperate release goal? The main reason why I think they should be merged is that the way to detect issues is similar. -- | Lucas Nussbaum | [EMAIL PROTECTED] http://www.lucas-nussbaum.net/ | | jabber: [EMAIL PROTECTED] GPG: 1024D/023B3F4F | -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Proposed new release goal: Dependency/file list predictability
Lucas Nussbaum wrote: acepack: r-cran-acepack's depends differ: -Depends: libc6 , libg2c0 , libgcc1 , r-base-core +Depends: libc6 , libgcc1 , libgfortran1 , r-base-core Caught by the BDFH. alamin: alamin-client's postinst differs: if [ -x /etc/init.d/alamin-server ]; then update-rc.d alamin-server defaults /dev/null if [ -x `which invoke-rc.d 2/dev/null` ]; then - invoke-rc.d alamin-server start || exit 0 + invoke-rc.d alamin-server start || exit $? Newer debhelper. ada-mode: files list differ: -./usr/share/doc/ada-mode/html/Using-non-standard-file-names.html +./usr/share/doc/ada-mode/html/Using-non_002dstandard-file-names.html WTF?? :-) af: files list differ: -./usr/lib/menu -./usr/lib/menu/af +./usr/share/menu +./usr/share/menu/af Non-ancient debhelper. -- see shy jo signature.asc Description: Digital signature
Re: Proposed new release goal: Dependency/file list predictability
On 22/06/07 at 20:08 +0100, Joey Hess wrote: Lucas Nussbaum wrote: acepack: r-cran-acepack's depends differ: -Depends: libc6 , libg2c0 , libgcc1 , r-base-core +Depends: libc6 , libgcc1 , libgfortran1 , r-base-core Caught by the BDFH. Ok, good to know. alamin: alamin-client's postinst differs: if [ -x /etc/init.d/alamin-server ]; then update-rc.d alamin-server defaults /dev/null if [ -x `which invoke-rc.d 2/dev/null` ]; then - invoke-rc.d alamin-server start || exit 0 + invoke-rc.d alamin-server start || exit $? Newer debhelper. ada-mode: files list differ: -./usr/share/doc/ada-mode/html/Using-non-standard-file-names.html +./usr/share/doc/ada-mode/html/Using-non_002dstandard-file-names.html WTF?? :-) af: files list differ: -./usr/lib/menu -./usr/lib/menu/af +./usr/share/menu +./usr/share/menu/af Non-ancient debhelper. Still, what do we want to do about those? -- | Lucas Nussbaum | [EMAIL PROTECTED] http://www.lucas-nussbaum.net/ | | jabber: [EMAIL PROTECTED] GPG: 1024D/023B3F4F | -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Proposed new release goal: Dependency/file list predictability
On Fri, Jun 22, 2007 at 01:49:25AM +0200, Steinar H. Gunderson wrote: as it's stable.) The idea is to build a package both in a pbuilder and in a really filled chroot -- it currently contains 18GB of packages, which is most of the devel and libdevel sections. What is compared is: This is exciting! Another evil idea that come to my mind is running piuparts preinstalling all packages that a package conflicts on, or replaces. This topic also triggers a continuous thought that I had since the RMLL conference in France, that is to use EDOS' computed SAT temperature of packages[1] to make testing even nastier. [1] SAT temperature: I'm not able to explain it precisely, but it's a number that ends up being proportional to how complex is the dependency tree of a package. Ideas that come to my mind are: - testing in piuparts *couples* of packages, both with high SAT temperature; - during testing, solving dependencies so that when a package depends on A | B | C, the package with the highest SAT temperature is pulled in; - test/audit the dependencies of high-SAT-temperature packages more throughly; - compute a delicateness index of a package by summing the SAT temperature of all packages that depend on it, and then give special attention to the most delicate packages to see RC bugs, maintainer activity and so on. I'm working on creating a sample data, but I'll be away for the week-end. Ciao, Enrico -- GPG key: 1024D/797EBFAB 2000-12-05 Enrico Zini [EMAIL PROTECTED] signature.asc Description: Digital signature
Re: Proposed new release goal: Dependency/file list predictability
Hi, Steinar H. Gunderson schrieb: To this end, I've set up the build daemon from Hell (BDFH) on my machine, currently doing script testing. Would it make sense to run the build under auto-apt, to see whether it tries to access some file in another package? That would obviously not be a replacement for the BDFH, since some particularly nasty tools install stuff into .d directories (so unless they are installed, they are never accessed), and it might generate a few false positives, but it could be another data source. Also, you could look at the atimes of the other packages after the build to see whether stuff has been used. Simon -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
