Bug#661078: britney: ignore additional packages in Sources index
On 2013-01-08 16:48, Ansgar Burchardt wrote:
Hi,
I finally got around to implement this in dak. The patch is available in
the f/extra-sources branch of my Git repository[1].
Sources only referenced by Built-Using will have a field
Extra-Source-Only: yes
Your demo seems to be using ExtraSourceOnly. :)
On franck.d.o you can look at the changes this would introduce to
testing's Sources with
cd /home/ansgar/public_html/root/srv/ftp-master.debian.org/ftp/dists
zdiff -d -u testing{-old,}/main/source/Sources.gz
The files are also accessible from [2].
Niels, could you update the patch to britney you prepared earlier?
Ansgar
[1] https://ftp-master.debian.org/users/ansgar/dak.git
[2] http://ftp-master.debian.org/users/ansgar/root/
Attached is the updated patch and the test-suite has been updated
accordingly.
~Niels
From fdf9089905f78f2e9f8253f60de5a5b723fb893b Mon Sep 17 00:00:00 2001
From: Niels Thykier ni...@thykier.net
Date: Fri, 29 Jun 2012 23:23:25 +0200
Subject: [PATCH] Ignore sources only referenced by Built-Using
Signed-off-by: Niels Thykier ni...@thykier.net
---
britney.py |3 +++
1 file changed, 3 insertions(+)
diff --git a/britney.py b/britney.py
index 0b03d30..7f56976 100755
--- a/britney.py
+++ b/britney.py
@@ -453,6 +453,9 @@ class Britney(object):
step = Packages.step
while step():
+if get_field('Extra-Source-Only', 'no') == 'yes':
+# Ignore sources only referenced by Built-Using
+continue
pkg = get_field('Package')
ver = get_field('Version')
# There may be multiple versions of the source package
--
1.7.10.4
Bug#661078: britney: ignore additional packages in Sources index
On 01/10/2013 05:57 PM, Niels Thykier wrote: On 2013-01-08 16:48, Ansgar Burchardt wrote: Sources only referenced by Built-Using will have a field Extra-Source-Only: yes Your demo seems to be using ExtraSourceOnly. :) Yep, I was confused as we use CamelCase in Release, but Words-with-Dashes in Sources. The test run was before I noticed this and changed the field name to the more appropriate Extra-Source-Only. Attached is the updated patch and the test-suite has been updated accordingly. Thanks. Then I have to annoy the wanna-build team next. packages.qa.debian.org, packages.debian.org and UDD (rmadison) might also want to ignore the new entries, but that shouldn't be a blocker. So I would like to enable the change once britney and wanna-build are patched. Ansgar -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#661078: britney: ignore additional packages in Sources index
On Thu, 2013-01-10 at 17:57 +0100, Niels Thykier wrote: On 2013-01-08 16:48, Ansgar Burchardt wrote: Sources only referenced by Built-Using will have a field Extra-Source-Only: yes [...] Attached is the updated patch and the test-suite has been updated accordingly. From fdf9089905f78f2e9f8253f60de5a5b723fb893b Mon Sep 17 00:00:00 2001 From: Niels Thykier ni...@thykier.net Date: Fri, 29 Jun 2012 23:23:25 +0200 Subject: [PATCH] Ignore sources only referenced by Built-Using Signed-off-by: Niels Thykier ni...@thykier.net Have a Signed-off-by: Adam D. Barratt a...@adam-barratt.org.uk to go with yours. ;-) Regards, Adam -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#661078: britney: ignore additional packages in Sources index
Hi,
I finally got around to implement this in dak. The patch is available in
the f/extra-sources branch of my Git repository[1].
Sources only referenced by Built-Using will have a field
Extra-Source-Only: yes
On franck.d.o you can look at the changes this would introduce to
testing's Sources with
cd /home/ansgar/public_html/root/srv/ftp-master.debian.org/ftp/dists
zdiff -d -u testing{-old,}/main/source/Sources.gz
The files are also accessible from [2].
Niels, could you update the patch to britney you prepared earlier?
Ansgar
[1] https://ftp-master.debian.org/users/ansgar/dak.git
[2] http://ftp-master.debian.org/users/ansgar/root/
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#657212: Bug#661078: britney: ignore additional packages in Sources index
Hi, Niels Thykier ni...@thykier.net writes: On 2012-02-24 01:22, Ansgar Burchardt wrote: I would like to include additional packages referenced by Built-Using in the Sources index[1] at some undefined point in the future. This might confuse britney which would need to just ignore them. Ansgar [1] http://bugs.debian.org/657212 In light of our IRC chat in #d-ftp today and the asumption that the Only-Extra-Source field will be implemented, I believe I have a trivial patch that will work[1]. There is an updated test for it in the britney2-tests[2] (the new repository announced today - not the old one). The patch is backwards compatible and could be applied before the extra sources appear in the Sources files (and without updating any of the unrelated existing tests). I remembered we don't have overrides and thus no Priority or Section fields for the extra sources. As even the current values for overrides aren't that useful (most source packages have Priority: source), I had an alternative idea: Instead of introducing a new field, use Priority: optional for regular sources and Priority: extra for the extra sources introduced by Built-Using. At least the latter would also go to Section: source. This would also fix #626394 (Priority: source in Sources). Ansgar -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#657212: Bug#661078: britney: ignore additional packages in Sources index
On 2012-07-31 09:30, Ansgar Burchardt wrote: Hi, [...] I remembered we don't have overrides and thus no Priority or Section fields for the extra sources. As even the current values for overrides aren't that useful (most source packages have Priority: source), I had an alternative idea: Instead of introducing a new field, use Priority: optional for regular sources and Priority: extra for the extra sources introduced by Built-Using. At least the latter would also go to Section: source. This would also fix #626394 (Priority: source in Sources). Ansgar Short: Sounds good to me. AFAICT, we don't use Priority for any thing in Britney, so it won't cause a lot of issues. The only thing is that Britney does not include Priority in the control files she writes. However, I believe she is the only consumer of those control files. If we assume Priority: optional if the field is missing, this should not be an issue[1]. If you change the value of the source field, that should also be fine. I see it is written to the HeidiResult file, but I think that particular column is stripped by another of our scripts before it is passed on (I think we use that column to strip out our faux packages). ~Niels [1] The filtering would ensure that she would never write a Priority: extra source in the control files, so the assumption should be reasonable. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#661078: britney: ignore additional packages in Sources index
On 2012-02-24 01:22, Ansgar Burchardt wrote:
Package: release.debian.org
Severity: wishlist
User: release.debian@packages.debian.org
Usertags: britney
I would like to include additional packages referenced by Built-Using in
the Sources index[1] at some undefined point in the future. This might
confuse britney which would need to just ignore them.
Ansgar
[1] http://bugs.debian.org/657212
Hi,
In light of our IRC chat in #d-ftp today and the asumption that the
Only-Extra-Source field will be implemented, I believe I have a
trivial patch that will work[1]. There is an updated test for it in the
britney2-tests[2] (the new repository announced today - not the old one).
The patch is backwards compatible and could be applied before the
extra sources appear in the Sources files (and without updating any of
the unrelated existing tests).
It is my understanding that the new field has only been proposed at this
point, but the field would greatly simplify the Britney changes and keep
the risks of regressions at a minimal.
~Niels
[1]
http://anonscm.debian.org/gitweb/?p=users/nthykier/britney.git;a=shortlog;h=refs/heads/bug-661078
(also attached)
[2] http://anonscm.debian.org/gitweb/?p=collab-maint/britney2-tests.git
From 9c3622ece11b53467c33bc1dab2dfa5c3216aebe Mon Sep 17 00:00:00 2001
From: Niels Thykier ni...@thykier.net
Date: Fri, 29 Jun 2012 23:23:25 +0200
Subject: [PATCH] Ignore sources only referenced by Built-Using
Signed-off-by: Niels Thykier ni...@thykier.net
---
britney.py |3 +++
1 file changed, 3 insertions(+)
diff --git a/britney.py b/britney.py
index 5205f64..e3e1cdb 100755
--- a/britney.py
+++ b/britney.py
@@ -453,6 +453,9 @@ class Britney(object):
step = Packages.step
while step():
+if get_field('Only-Extra-Source', 'no') == 'yes':
+# Ignore sources only referenced by Built-Using
+continue
pkg = get_field('Package')
ver = get_field('Version')
# There may be multiple versions of the source package
--
1.7.10
Bug#661078: britney: ignore additional packages in Sources index
On 2012-02-24 01:22, Ansgar Burchardt wrote:
Package: release.debian.org
Severity: wishlist
User: release.debian@packages.debian.org
Usertags: britney
I would like to include additional packages referenced by Built-Using in
the Sources index[1] at some undefined point in the future. This might
confuse britney which would need to just ignore them.
Ansgar
[1] http://bugs.debian.org/657212
Hi Ansgar,
I believe Britney already handles this reasonably[1]. Of course, I
assume here that by ignore you mean it is (also) okay for Britney not
to include them in the resulting HeidiResult file.
Assuming there will be a source package in Sources only kept due to
Built-Using, then Britney may (or may not) keep its counterpart in
testing until the source package disappears from Sources (in sid). If
it does not have any binaries and it is not in testing, then she will at
least keep it out of testing[2].
It is possible I am missing a corner case or two, so I am keeping the
bug open for now.
~Niels
[1]
while step():
pkg = get_field('Package')
ver = get_field('Version')
# There may be multiple versions of the source package
# (in unstable) if some architectures have out-of-date
# binaries. We only ever consider the source with the
# largest version for migration.
if pkg in sources and
apt_pkg.version_compare(sources[pkg][0], ver) 0:
continue
[2] Example from last nights excuses:
libffi-ruby (- to 0.6.3debian-2)
Maintainer: Micah Anderson
516 days old (needed 10 days)
libffi-ruby has no binaries on any arch
libffi-ruby (source) has new bugs!
Updating libffi-ruby introduces new bugs: #652755
Not considered
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#661078: britney: ignore additional packages in Sources index
Niels Thykier ni...@thykier.net writes: I believe Britney already handles this reasonably[1]. Of course, I assume here that by ignore you mean it is (also) okay for Britney not to include them in the resulting HeidiResult file. That is fine. Assuming there will be a source package in Sources only kept due to Built-Using, then Britney may (or may not) keep its counterpart in testing until the source package disappears from Sources (in sid). If it does not have any binaries and it is not in testing, then she will at least keep it out of testing[2]. dak would also include them in Sources for testing, britney could not remove them as they are handled differently in dak. Would britney handle that as well? For reference here are the sources that would currently be included in Sources right now: WITH extra_sources AS (SELECT DISTINCT es.src_id FROM extra_src_references es JOIN bin_associations ba ON es.bin_id = ba.bin WHERE ba.suite = 4 EXCEPT SELECT source FROM src_associations WHERE suite = 4) SELECT source, version FROM source WHERE id IN (SELECT * FROM extra_sources); For testing (suite=4): gcc-4.6 | 4.6.2-6 binutils | 2.22-2 gcc-4.6 | 4.6.2-1 gcc-4.6 | 4.6.1-16 binutils | 2.22-1 gcc-4.6 | 4.6.2-5 For unstable (suite=5): gdb | 7.3-1 gcc-4.6 | 4.6.0-9 gcc-4.6 | 4.6.2-14 binutils | 2.22-1 Regards, Ansgar -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#661078: britney: ignore additional packages in Sources index
On Fri, 2012-02-24 at 09:01 +0100, Niels Thykier wrote: I would like to include additional packages referenced by Built-Using in the Sources index[1] at some undefined point in the future. This might confuse britney which would need to just ignore them. [...] I believe Britney already handles this reasonably[1]. Of course, I assume here that by ignore you mean it is (also) okay for Britney not to include them in the resulting HeidiResult file. [...] [1] [...] # There may be multiple versions of the source package # (in unstable) if some architectures have out-of-date # binaries. We only ever consider the source with the # largest version for migration. This results in britney's internal list of source packages not including the older versions, so I'm not convinced they'd actually end up in the Heidi at all. Another fun case might be if the older version included binaries which are no longer built by any source. It's possible I just need more coffee and this all just works[tm] of course... :) Regards, Adam -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#661078: britney: ignore additional packages in Sources index
On 2012-02-24 09:33, Ansgar Burchardt wrote: Niels Thykier ni...@thykier.net writes: [...] Assuming there will be a source package in Sources only kept due to Built-Using, then Britney may (or may not) keep its counterpart in testing until the source package disappears from Sources (in sid). If it does not have any binaries and it is not in testing, then she will at least keep it out of testing[2]. dak would also include them in Sources for testing, britney could not remove them as they are handled differently in dak. Would britney handle that as well? I think so. If there are two sources with different version, she completely ignores the lowest version. So that is definitely not a problem. Britney has her own copy of testing/Sources, which she maintains (removing fields she doesn't care about etc.). So sources injected by dak wouldn't be noticed unless we refresh those sources. I admit I do not know if we do that regularly. That being said, the sources (re)appearing like this would (as far as I can tell) basically fall into three cases: 1 lower version than source of same name (in same suite) = ignore/discard (see above) 2 lower version in testing than source of same name in sid = attempt to migrate sid version (as usual) 3 source only in testing or (at end of run) source has no binaries in testing = attempt removal from testing I suspect 2 might happen if we remove something from testing. Though the sid version probably won't migrate then. To me, it looks like they will be reduced to a previously known solution. Adam, can you see another case for the reappearing sources? For reference here are the sources that would currently be included in Sources right now: [...] Regards, Ansgar ~Niels -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#661078: britney: ignore additional packages in Sources index
tags 661078 confirmed thanks On 2012-02-24 10:03, Niels Thykier wrote: On 2012-02-24 09:33, Ansgar Burchardt wrote: [...] I think so. If there are two sources with different version, she completely ignores the lowest version. So that is definitely not a problem. Britney has her own copy of testing/Sources, which she maintains (removing fields she doesn't care about etc.). So sources injected by dak wouldn't be noticed unless we refresh those sources. I admit I do not know if we do that regularly. That being said, the sources (re)appearing like this would (as far as I can tell) basically fall into three cases: 1 lower version than source of same name (in same suite) = ignore/discard (see above) 2 lower version in testing than source of same name in sid = attempt to migrate sid version (as usual) 3 source only in testing or (at end of run) source has no binaries in testing = attempt removal from testing [...] IRC chatting revealed at least one issue. Built-Using can make a newer source version appear in testing. build foo against bar 2. bar 1 is in testing. put foo in testing. The current suggestion is to use the Packages_X files to determine which source versions to use (and ignore the rest). Though: 09:21 adsb does it dtrt with smooth updates and/or faux packages? (not expecting an answer right now, just a thought) [...] 09:22 adsb it = inferring Sources from Packages As always, test cases are welcome at [1]. ~Niels [1] http://anonscm.debian.org/gitweb/?p=collab-maint/britney-tests.git -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#661078: britney: ignore additional packages in Sources index
Package: release.debian.org Severity: wishlist User: release.debian@packages.debian.org Usertags: britney I would like to include additional packages referenced by Built-Using in the Sources index[1] at some undefined point in the future. This might confuse britney which would need to just ignore them. Ansgar [1] http://bugs.debian.org/657212 -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
